Cleaning Robot with Hand Gesture

Cleaning Robot with Hand Gesture

Aishwarya Sreedhar, Sunanda, Yashawini A. D.

B.E. in Electronics and Communication Engg NIE Institute of Technology Mysuru, Karnataka, India

Abstract – Gesture and gesture recognition terms are heavily encountered in human computer interaction. Gesture are motions of the body or physical action form by the user in order to convey some meaningful information. Gesture recognition is the process by which gesture made by the user is made known to the system. Hand gesture has the natural ability to represent ideas and actions very easily. Thus, using these different hand shape, being identified by gesture recognition system has potential to provide a more natural interference to the computer system.

The conventional floor cleaning machines is most widely used in airport platform, railway platform, hospitals,bus stands, malls and in many other commercial places. These devices need an electric supply for its operation and not user-friendly. In India, especially in summer, there is power crisis and most of the floor cleaning machine is used effectively due to this problem. Hence, it is need to develop low cost, userfriendly floor cleaning machine. In this paper, an effort has been made to develop a gesture operated floor cleaning machine so that it can be an alternative for conventional floor cleaning machine.

Keywords-Gesture, Gesture recognition, human computer interaction.

1. INTRODUCTION

   In the present day framework of interactive, intelligent computing, an efficient human-computer interaction is assuming at-most importance. Gesture recognition can be termed as an approach in this direction. It is one of the essential techniques to build user-friendly interfaces. It is process by which the gestures made by the user are recognized by the receiver.

   Gesture are expressive, meaningful body motions involving physical movements of the fingers with

   the intent of conveying meaningful information and interacting with the environment. Through the use of computer vision or machine eye, there is great emphasis on using hand gesture as a substitute of new input modality in board range application. With the development and realisation of virtual environment, current user-machine interaction tools and methods including mouse,joystick,keyboard and electronic pen are not sufficient. Hand gesture has the natural ability to represent ideas and actions very easily. Thus,using these different hand shapes, being identified by gesture recognition system and interpreted to generate corresponding event, has the

   potential to provide a more natural interface to the computer system. This type of natural interaction is the core of immersive virtual environments. If we ignore a world of computer for a while and consider interaction among human beings, we can simply realise that we are utilising wide range of gestures in our daily personal communication. By the fact, it is also shown that people gesticulate more when they are talking on telephone and are not able to see each other as in face to face communication. Gestures very greatly among cultures and context still are intimately used in communication. The significant use of gestures in our daily life as a mode of interaction motivates the use of gestural interface.

   Gesture is one of the natural base of Human Computer Interaction(HCI) and it has intuited controls method. In general, gestures are defined as motions involving hands and combinations of hand and arm. It is usually divided into two types:

   Static and dynamic gesture. Static gesture means the shape of single hand, which is corresponding to the relevant points in the model-parameter space. While dynamic gesture is composed of a series of motion, corresponding to a trajectory in parameter space, which is described by the space characteristics changing with time. Algorithm on static gesture recognition have developed rapid in recent years such as gesture recognition based on artificial neural network and computer vision.

   However, simple static gesture cannot meet the requirement of the industry application and methods on dynamic gesture recognition havebecome a focus in the area of research instead. Owing to the diverse city and complex city of gesture and flexible position and shape of hand, it makes gesture recognition become multi discipline and challenging project[1][2][3][4].

   This paper is organised as follows:Section II describe the related work. Section III brief about existing system. Section IV describes the proposed system. Section V has the system architecture. Section VI explains about the working of the project.

2. RELATED WORKS

   This chapter provides an overview of the state of the art with survey of existing techniques, object distance measurement and servoing. The main goal of this chapter is to give the reader a comprehensive knowledge of previous studies and technical information about the above-

   mentioned techniques. This chapter also classifies the techniques and then presents the strategies efficiently tackle the challenges of object tracking and visual servoing.

   Detect and track the desired object:

   Object detection is required by every tracking method to initialise the tracker and to detect the existence of known objects in a given image frame. However,in this paper, object detection is described as detecting the existence of any object in the field of view that can be potentially manipulated. Since the main goal of the proposed algorithm is to analyse the movement of object, object tracking is one of the systems essential parts. In the real world, an object can move freely in a 3-D space with 6 degrees of freedom(DOF). Due to the complexity of the problem, we divide the tracking problem to two sub- problems:object point(closest point of the object to the camera)tracking, which is related to the objects movement in xy-plane; and estimation of the objects orientation, which objects movement about the z-direction.

   Object tracking:

   Object tracking is a challenging problem in robot vision due to various factors such as camera motion, occlusions, non-rigid object structures and unanticipated rapid changes in both the object and the scene.

   Object tracking is relevant in the following topics [9]. Surveillance systems for monitoring a behaviour or changing information in order to detect unusual activities. Video indexing for retrieval and recovery of videos in databases.

   Traffic monitoring for simultaneously traffic inception to direct traffic flow.

   Vehicle navigation for real-time path planning and obstacle avoidance capabilities in robotics.

   Object tracking problems:

   Tracking a desired object can be a complex task. Since, estimating the trajectory of the moving object in the image plane has to be accurate. The main problems related to object tracking are as follows[9]:

   Information loss when projecting for 3-D to 2-D image. Existence of noise in the images.

   Complex motion of the objects.

   Complex characteristics of the objects such as non-rigid/ articulated objects.

   Partial or full object occlusion. Complex shapes of the objects. Illumination variance in the scene. Real-time processing requirements.

   Traditional hand gesture recognition research began in the 1980s.

   Paper [5] proposes a static recognition using image- processing based on words in ASL (American Sign Language). This method can accurately identify the 27 of the 31 words in the sign language.

   Paper [6] proposes the recognition of ASL by extracting the features the image as hidden Markov model.

   Paper [7] came up with amethod about recognising gesture for video content navigation.

   Paper [8] proposes method based on colours of skin.

3. EXISTING SYSTEM

   The existing robotic vacuum cleaners and industrial cleaning machines are available in the market but only few ones implement wet cleaning of floors. There are many various types of cleaning machine or equipment on the market today for both commercial and residential use. The introduction of cleaning machine made the janitors life easier and increase cleaning efficiency. However, manual agitation is burdensome to the person cleaning and increase labour. Thisunit are normally electric plug powered and hard to reach smaller area. The standard method for wet floor cleaning relies on sweeping the floor and thereafter running a damp cloth or a mop across the swept floor. While tried and true, this method is time consuming and labour-intensive which results in scheduled floor cleanings being skipped for other less onerous task. In order to overcome the time and labour intensity associated with floor cleaning, automated floor cleaning systems have been proposed. Such system generally falls into one of the two categories the first type of such system relies on various sensors and artificial intelligence in order to systematically move the floor cleaner across the floor. The sensor detect object and other obstruction and artificial intelligence system determines the force of action of the device with respect to the object or obstruction.

4. PROPOSED SYSTEM

   Floor cleaner along with hand gesture is a compact robotic system which provides floor cleaning service in room and big offices reducing human labour. Basically as a robot, it eliminates a human error and provide cleaning activity with much more efficiency. Also, due to manual involved, it is time consuming and irritating to clean the floor. In big offices, the floor area is very huge and people involved there for cleaning purposed cannot clean it more efficiently. This is where robot with hand gesture comes has an advantage being small and compact in size so, we can carry it and place it where ever we want. Also, in industries, the robot is very cost effective as compare to manual labour involved. The flexibility, time saving and efficiency along with gesture makes the robot perfectly clean choice for cleaning the floor.

   Human gesture interpretation by a computer is utilisehuman-computer interaction in the area of computer vision [10]. The of gesture recognition is to recognise a specific human gesture and convey information to the user pertaining to individual gesture. A particular gesture of interest can be recognised from collection of gesture and depending on that, robotic arm is given particular command for execution [11]. Human body language recognition using computer is overall aim of project, thus making a link between using human-machine interaction. Human machine interaction can be enhanced by hand

   gesture recognition without relying on convention input devices such as key broad and mouse. Tele robotic control applications utilise extensive use of hand gesture [10] [12]. It can control naturally and intuitively a robotic system. Geometrical information to the robot can be sent in natural way which is dominant benefit of such system. Hand gestures can be very well recognised by glove based techniques.

   This cleaning robot can recognise hand gesture, can take command from humans and for those who are unable to speak or hear, having a robot that can recognise sign language would allow them to communicate with it.

   IR SENSOR

   IR SENSOR

5. SYSTEM ARCHITECTURE

   Fig 1 shown above is the block diagram of cleaning robot with hand gesture.

   1. Hardware description:

      Microcontroller(STM32) is used which is provided with clock signal (quartz crystal operating at 72MHz frequency). DC motors attached to motor driver to provide high current and most importantly it is installed with ultrasonic sensor and IR sensors. For power supply to separate batteries are used. Bluetooth is used for wireless communication between computer and robot.

   2. Software description:

   L293D MOTOR DRIVER

   L293D MOTOR DRIVER

   Fig 2 shown below is the flow chart of software operation. The MATLAB program is require for the functioning and usage of gesture. To interface the acquired gestures and the robot, Arduino is required a code is written in Arduino IDE to run the robot.

   ULTRA SONIC SENSOR

   ULTRA SONIC SENSOR

   STM-32

   BLUETOOTH

   DC MOTORS

   BLUETOOTH

   DC MOTORS

   PUMP DRIVER

   Fig 1: Block Diagram

   PROCESSING IMAGE(MATLA

   PROCESSING IMAGE(MATLA

   RS-232

   COMMUNICATION

   RS-232

   COMMUNICATION

   STM

   MOVEME

   STM

   MOVEME

   IMAGE DECISION MAKING ACQUISITION B)

   (MATLAB)

   CLEANING ROBOT -32

   NT CONTROL

   Fig 2:

   IMAGE DECISION MAKING ACQUISITION B)

   (MATLAB)

   CLEANING ROBOT -32

   NT CONTROL

   Fig 2:

   TURN ON(Gesture)

   TURN ON(Gesture)

   START

   MOVING=true

   MOVING=true

   If obstacle Or height

   Turn to avoid obstacle

   Turn to avoid obstacle

   OFF (Gesture, if area is cleaned)

   OFF (Gesture, if area is cleaned)

   YES

   STOP

   Fig 3: Flow Chart

   OBJECT DETECTION

   Software Operation

   NO

   SL.NO.	REQUIRMENT	SPECIFICATION
   1.	BATTERY LIFE	3.77 HRS
   2.	WATER TANK	½ LITERS
   3.	WEIGHT	2.5 Kg
   4.	CLEANING TIME	AS PER THE AREA
   5.	SOFTWARE	MATLAB ARDUINO IDE
   6.	COMMUNICATION PROTOCOL	UART
   7.	POWER OF MOTOR	7.2W
   8.	CURRENT COSUMPTION	600mA

   Table-1: Software and Hardware Description

6. WORKING

   The cleaning robot is intelligently programmed to clean a large area using a mop. The cleaning robot is cost effective, convenient, environment friendly that saves the valuable time of any person.

   The robot is made on a square piece of broad that has 4 wheels beneath it and brush attach to one its side in order to clean as it passes over the surface. DC motor is used for the movement of the wheels which is connected to the platform. Two motor drivers are required so as to provide it a high current i.e. up to 0.72 to 1.2 Ampere. The driver used is named as L293D with H- bridge configuration. The robot is handy and can rotate anywhere in any direction by providing appropriate gesture.

   Sensors are basically used to setup a communication link between the outside world and digital device and to fulfil the purpose use of ultra-sonic sensors are incorporated to detect the obstacles in front of the robot so it stops. IR sensor detect the height in order to prevent the robot from falling down and changes its direction accordingly. A water tank is placed on top of the platform used to spray water while cleaning the surface.

7. CONCLUSION

The research facilitates efficient floor cleaning. Since, in the project the robot is incorporated with different devices like DC motors, ultrasonic sensors, IR sensors, Bluetooth along with gestures from a digital device provided by human, so it will be easy to handle, save time and work according gestures for cleaning purpose at offices and industries.

With simple algorithm and program code, the robot will be able to cover large floor areas.

In everyday life, cleaning is the most important as well as difficult task. The existing system is more laborious and demanding work. Therefore, cleaning robot wih gesture is design to make cleaning process easier rather than using manual technique. Households of today are becoming smarter and more automatic. The purpose of robot is to reduce the labour and time, less in the physical activity as well as keep a check on expenditure in big industries and large area. The main objective of this proposed system is to make cleaning process easier and effective by a single human intervention using hand gesture in more innovative technology. Most importantly, it provides direct and effortless communication between human and robot. This robot cannot only be run by operator but also operated by other masses. Mainly, as it uses gestures, it becomes helpful for people who are unable to speak and hear to work easily with sign language. It can be mainly used in commercial field as well as domestic field.

REFERENCES

1. Binghai Ren, YuanxinGuang et al. Vision- Based Recognition of Hand Gestures. Chinese Journal of Electronics 2000,28(2):118-121DZ.

2. oushu Hu, Survey of gesture recognition technology. China Science and technology information, 2005, 2:41-42DZ.

3. ZhiQuan Feng, Yan Jiang. A survey of Hand Gesture of recognition. Journal of University of Jinnan2013,27(4): 336-341DZ.

4. S.S. Rautaray, A. Agrawal. Vision based hand gesture recognition for human computer interaction: a survey. Artificial Intelligence Review,2012, DOI 10.1007/s10462-012-9356-9.

5. C. Charayaphayan, A.Marble. Image processing system for interpreting motion in American sign language. Journal of Biomedical Engineering, 1992, 14:419-425.

6. T.Starner et al. Visual recognition of American Sign Language using hidden markov models. MIT Media Laboratory, MIT, USA,1995.

7. G. Bradski, B. L. Yeo, M. M. Yeung. Gesture for video content navigation. Proceeding of Storage and Retrival for Image and video Database VII, 1999:230- 242.

8. M. J. Jones, J. M. Rehg. Statistical colour model Models with Application to Skin Detection. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR99), 1999,1(1):1274.

9. T. Starner, J. Weaver, A. Pentland. Real-time American Sign Language recognition using desk and wearable computer based video. IEEE transactions on pattern Analysis and Machine Intelligence, 1998, 20(12): 1371-1375.

10. Ng, Chan Wah, and Surendra Ranganath. Real-time gesture recognition system and application. Image and Vision computing 20.13 (2002): 993-1007.

11. Ramamoorthy, Aditya, et al. Recognition of dynamic hand gesture. Pattern Recognition 36.9(2003): 2069- 2081.

12. Raheja, JagdishLal, et al. Real-time robotic hand control using hand gesture. Machine Learning and Computing (ICMLC),2010 Second International Conference on. IEEE, 2010.