Vision-Based Fallen Identification and Hazardous Access Warning System of Elderly People to Improve Well-Being

Vision-Based Fallen Identification and Hazardous Access Warning System of Elderly People to Improve Well-Being

W. S. L. Abeyrathne, B. G. D. A. Madushanka, H. D. N. S. Priyankara

Department of Mechanical Engineering, The Open University of Sri Lanka, Sri Lanka

Abstract In recent years, fall recognition, and access limitation has been a challenging issue for elders and patients. The critical attribute of this research is to support the healthcare system and hence the growth in the elderly population. The need for fall warning equipment and sensors has also risen as populations have grown because it improves the lives of elderly caregivers and patients. This paper reveals a different form of detection of fall and newly discovered region, restriction of entry to mitigate the injuries that might occur. They are attempting to classify the different forms of falls that are deemed harmful. And try solutions to detect crashes utilizing tracking sensors, Ambience sensors and Sight related apps. Within this paper, it was addressed the fall identification approaches utilizing vision-based program and access limitation by first utilizing vision and then restricting by alarms and lights.

Keywords Fallen identification, Access restriction, Elderly people, Accidents, Vision-Based, Warning system, Well-being

1. INTRODUCTION

   Support to the aged and disabled patients have recently become an important subject. Since the elderly population has increased, their healthcare has become essential. This problem is a very daunting issue in places like Japan, Italy, and Germany etc. The elderly is now seeing a significant increase worldwide. More than 11.5% of the world's people are aged 65 and older in 2012, and by 2050 it is predicted to hit 2 billion [1]. The problem raised by these figures is the care of the people who prefer to save their independent lifestyles, often at high risk. Also, extra care homes are commonly used to provide protection, reassurance, and health support for the elderly living in these facilities. Within such an environment, older people feel more comfortable conducting their day-to- day tasks while a dedicated team offers 24/7 emergency care services. Disable patients also a vital subject in the field of healthcare. According to the World Health Organization (WHO), over one billion people globally experience disabilities. In Sri Lanka, there exist thousands of disabling patients who are suffering from varieties of disabilities [3]

   This trend is widely endorsed by developed country governments such as Singapore, Germany, Scandinavians, etc., and affluent countries such as Saudi Arabia, Kuwait, Qatar, Oman, etc. (now they focus entirely on third-world workers). Hundreds of small conferences have been held to help this trend. Still, mostly in 2008, the government of Singapore arranged a conference and exhibition on the declines of older people and patients (with no limits on access) [6]. While wearable sensors and actuators do the job, elders

   and patients hesitate to wear them because they are nervous and do not want to get support from others.

   Some work has been done, and several initiatives to address this issue. The researchers cut this field down for their desire to relax. Hence, creating a modern advance for fall detection is simple for the latest researches.

   Many efforts have made in fall identification due to its significant demand and their big potential market and most importantly, the social value. A series of different technologies have been developed to eradicate this problem. I have made a chart that shows the technologies that have proposed to solve the problem [4].

   Overcome this problem, and it was suggested a machine vision system to monitor the status of the elderly or patients and warn the elder if they are trying to enter a restricted area. Following shows the procedure for the study. For the wearable device type, in early stages, a walking stick was made with a sensor installed inside to detect the fall. It used a gyroscope to detect the angular velocity of which the stick falls and the predefined velocity shows the stick has fallen. Moreover, that leads to the assumption the walking stick holder to is fallen. This system had many defects, as not all the elders use a walking stick and sometimes the fall of the stick alone cause lead to misunderstandings. Therefore, the idea was not successful [9].

   Fig. 1: Wearable sensors

   There were some other wearable devices, as shown in Fig. 1, the most common idea of most of them were to wear an accelerometer to detect the fall by the motion. Some of the wearable devices used the vibration caused by the fall to detect the fall.

   There were some advantages of wearable devices, as all the sensors were cheap and most importantly, they were straightforward to set up and operated. The main disadvantage of the wearable devices is the motion of the devices are detected relevant to the earth without relation to the wearer. So that lots of false alarms were made [9]. The most prominent opponent of the wearable devices was the unwillingness of the elders to wear the device; they all think that they can handle themselves without any outside help. Fig. 2 is shown that the non-wearable sensor platform when elders and patients refuse to wear them as its uncomfortable and they do not like to receive help from others.

   The next approach was the ambience device. It is the use of multiple sensors to collect data about the humans behaviour. Use of vibration sensors to detect the vibration when the human falls on the ground, which is shown in Fig. 3. By fixing a few sensors of that, the type could analyze the location of the human also. Another method of this manner was to fix a sensor to detect if the human leaves the bed; it was a bit narrow view. Another method used was using a pyroelectric IR sensor array on the wall for human activity identification [12]. All those methods were theoretically correct, but in practice, all those showed lots of errors and false alarms [5].

   Fig. 2: Non-wearable sensors

   Fig. 3: Ambient sensors

   So, to minimize the number of errors and increase the accuracy of the result, it was thought of choosing the vision- based system to Detect the fall of elderly. According to researchers, also, the accuracy of vision-based systems was higher than in other ways [7]. Therefore, to overcome this, it was suggested a machine vision system to monitor the status of the elderly or patients and warn the elder if they are trying to enter a restricted area. Firstly, camera vision was used for fundamental operations, the video was taken a life, and an operator was placed to monitor the video. If there is a problem with the person in the video, he is to detect it and come to assumptions, whether the person in the video needs any help or assistance or nothing wrong with the person and just ignore it. That system entirely relies on the accuracy and the performance of the operator. Today machine vision systems continue to move forward. 3D vision systems that scan products running at high speeds are becoming affordable, and systems that do everything from thermal imaging to slope measurement can be readily found. Machine vision continues to be a growing market, with many new advances driven by the vast array of possible applications [8] [10] [11].

2. COMPONENT SELECTION

   All the components used in this machine vision system were selected on some valid reasons. The main components that were used in the system are camera, Arduino Nano board, power adaptor, Laptop, GSM module and warning lights. For this scenario, the Kinect camera is used to capure human fallen detection. Activate a siren, and the easiest method is to use an Arduino so that lots of wiring and components could be minimized rather than using relays and other circuits. Nano board was chosen as not a lot of outputs are to be controlled.

   A. Web camera

   Web camera of the Laptop is used, as the connection with the Laptop is easy this way. It has no difference than using an external camera for the capturing of the elders movement. As the resolution of the web camera is 1080*720, it is well enough to identify the movement.

   1. Arduino UNO Board

      The Arduino UNO 2560 is designed for projects that require more I/O lines, more sketch memory, and more RAM. With digital I/O pins, 16 analogue inputs. It is the recommended board for 3D printers and robotics projects. It contains everything needed to support the microcontroller; simply connect it to a computer with a USB cable or power it with an AC-to-DC adapter or battery to get started, and it is used to activate the siren and activate the GSM module when need.

   2. Alarm System

      This device is used to warn the elderly if he approaches a restricted area (door, corridor). And if the elderly falls and the unit confirm it, the alarm should be activated. This unit should have a manual switch off button also.

   3. GSM Module

      The device is used to receive a signal from the Arduino and, send the message to the mobile phone for proper service.

      Information management and communication present an ultra- compact and reliable wireless module-SIM900. The Quad- band GSM/GPRS module in an SMT type and designed with a potent single-chip processor integrating ARM926EJ-Score, allowing you to benefit from small dimensions and cost- effective solutions.

3. SUGGEST DESIGN AND IMPLEMENTATION When a human enters the room, the program should be

   able to identify the human first. Within the room, the program will be able to identify the human, although the height and the width is a bit different when moving away from the camera. So, the height and width range was given.

   A logical system in the vision system was created to identify the pose/status of the older person and is to identify and continue the identifying process throughout the whole time. Python with open cv libraries is going to be used for this process, which is the most critical and time-consuming part. First, video processing should be done to a saved video, then live video retrieval is to be done.

   After a live video is uploaded to the program, human and object identification is to be done. As the video resolutions affect the efficiency of providing an output, the video resolutions are reduced. Then to detect the edges more quickly and accurately, the colour is converted to black and white. This video is used to track the humans by identifying the motion, which is done by subtracting consecutive frames (old from new). Then if there is a difference among them, it is considered as a motion. The proposed system detects only human, and any background information such as electric equipment, pets, and objects are not detected.

   First, the Machine vision system has consisted of a Web camera, Laptop, Program, and alarming system. The Web camera does the video retrieval part, and then the video data is sent to the computer program to extract the required data and conclude a fallen state and alarming state. Then the signal is sent to the Arduino to send an SMS or warn the elder.

4. METHODOLOGY

   This smart system is cost-effective and provides quick accident detection and notification. System implementation involves three Sectionsinitially the identification of the human pose.

   1. Human detection and fallen identification

      Fig. 4: Flow chart for the pose identification

      Fig. 4 represents the flow chart of the pose identification, which leads to the identification of a fall.

   2. Sending an SMS if the elder is identified Fallen

      First, the system identifies whether it is a human or not, then identification of the pose as fallen or standing by the algorithm.

      When the system identifies the elder as fallen, it sends a signal to the Arduino which activates the GSM module to send a predefined SMS to the caretakers. The scenario is given as a block diagram in Fig. 5.

      Fig. 5: The block diagram of the GSM module activation

   3. Area restriction and warning system

   Fig. 6: Flow Chart of area restriction warning system

   To identify the restricted area, it used a non-disturbing sticker in it, and it is quite durable in my system as the predefined restricted areas do not change, e.g.: – door, window. The camera is fixed in the top the room and directs to the door. This process is described in a flow chart in Fig 6.

   Fig. 7: Elder standing near the restricted door

   Fig. 7 shows the elder is standing near the restricted area. The older person is standing in front of the door and try to open it. His body covers the red mark on the door. Then the system detects the obstruction of it.

   Fig. 8 shows the elder is standing in front of the door and 2 meters away. As the camera can see the marker on the door, it considers as the elder is not near the restricted area. The older person is standing in front of the door and away 2 meters. His body does not cover the red mark on the door. Then, the system detects that the elder is safe without triggering the alarms.

   Fig. 8: Elder standing away from the door

   Fig. 9: The elder is standing in front of the door but 2 meters away

   Fig. 9 shows the elder is in front of the door and away from the door. When the elder walks near the restricted area, the market is closed to the camera fully or partially. That is when the program identifies that the elder is near the restricted area. Then it sends an activation signal to the Arduino to activate the warning lights. Block diagram of the system is shown in Fig. 10.

   Fig. 10: System overview

   Fig. 10 represents the process of warning bulb activation and SMS sending when the processor decides. The elder is fallen or trying to access a restricted area. Fig. 11 is shown the process of activating the warning bulbs is described.

   Fig. 11: Block diagram of the restriction warning system

5. SYSTEM OVERVIEW

   The system is to retrieve the video using the web camera. This video is directly transmitted to the Laptop, which runs the program of fall identification. Once the video is uploaded, the program detects the human and does the video processing part, e.g.: -noise reduction, filtration and fall identification. It waits until the program detects the elder was entering a restricted area if so, the signal is sent to the Arduino to actuate the siren. If the elder is identified fallen, the pre-programmed text is sent to the pre-entered mobile number of the caregiver.

6. RESULTS

   Fig. 12, 13, 14, 15, and 16 are shown the identification of different poses, respectively.

   Fig. 12: Human detection

   Fig. 13: Pose identification (standing)

   Fig. 14: Pose identification (fallen pose 1)

   Fig. 15: Pose identification (fallen pose 2)

   Fig. 16: Pose identification (fallen pose 3)

   Fig. 17: SMS sent to the caretakers about the fall

   Fig. 17 shows the SMS screenshot is sent to the caregiver where any of fallen pose occurs is shown in Fig. 14, 15, and 16, respectively.

   Fig. 18: Human standing safely in front of the restricted area

   Fig. 19: Human standing away from the door

   Fig. 20: Human standing in front of the door

   The Fig. 18,19 and 20 show the elders movement around the restricted door and the system identifying whether the elder is trying to open the door or just moving safely.

7. CONCLUSION

This research was to identify a fallen elder and, inform the responsible parties about it, and secondly warn the elder and caretakers if he tries to enter to dangerous areas. Also, this research is shown the ways toidentify the standing and fallen poses of the elder, which could help the society in elder protection. First, there were few methods to identify the falls when sudden changes in the movement. Second is to identify

the height difference (height difference between head to knee, when standing and falling). Finally, the proposed system is used to identify the area of the contours made by the elder and that was used to identify the pose continuously. High accuracy result was obtained within a few seconds until the warning message sending to the caregiver for his proper service. The informing methods could be improved in the future by saving a video, audio or sending a live video to the user when a fall happens or else. But there are security and privacy issues in live streaming, so that should be considered. The Kinect camera was used to measure the distance between objects. A high-performance processor is helped to process the live video quickly and process the sudden movements accurately.

REFERENCES

1. Yu, X. (2018). Approaches and principles of fall detection for elderly and patient – IEEE Conference Publication. [online] Ieeexplore.ieee.org.

2. Rajagopalan, R., Litvan, I. and Jung, T. (2018). Fall Prediction and Prevention Systems: Recent Trends, Challenges, and Future Research Directions.

3. Herath, Kasun & de Mel, W.R.. (2019). ELECTROENCEPHALOGRAM (EEG) CONTROLLED ANATOMICAL ROBOT HAND. 10.6084/m9.figshare.9759248.

4. Harrou, F., Zerrouki, N. and Sun, Y. (2018). Vision-based fall detection system for improving the safety of older people – IEEE Journals & Magazine.

5. Khawandi, S., Daya, B. and Chauvet, P. (2018). Implementation of a monitoring system for fall detection in elderly healthcare.

6. B Bonnechère1, B Jansen2,3, P Salvia1, H Bouzahouene1, L Omelina2, J Cornelis2, M Rooze1, S Van Sint Jan Laboratory of Anatomy, Biomechanics and Organogenesis (LABO), Université Libre de Bruxelles,Brussels, BELGIUM

7. J. Baek and B. J. Yun, Posture monitoring system for context- awareness in mobile computing, IEEE Trans. Instrum. Meas., vol.59, no. 6, pp.1589-1599, 2010.

8. Y. Tao and H. Hu, A novel sensing and data fusion system for 3-D arm motion tracking in telerehabilitation, IEEE Trans. Instrum. Meas., vol. 57, no. 5, pp.1029-1040, 2008.

9. M. Mubashir, L. Shao, and L. Seed, A survey on fall detection: principles and approaches, Neurocomputing, vol. 100, pp. 144-152, 2013.

10. W. Y. Shieh and J. C. Huang, Falling-incident detection and throughput enhancement in a multi-camera video-surveillance system, Medical Eng. and Physics, vol. 34, no. 7, pp.954-963, 2012.

11. S-G. Miaou, P-H. Sung, and C-Y. Huang, A customized human fall detection system using Omni-camera images and personal information, in Proc. IEEE 1st Transdisciplinary Conf. Distributed Diagnosis and Home Healthcare, pp. 39-42, 2006.

12. B. Jansen and R. Deklerck, Context-aware inactivity recognition for visual fall detection, in Proc. IEEE Pervasive Health Conf. and Workshops, pp. 1-4, 2006.

13. Herath, Kasun. (2016). RICE GRAINS CLASSIFICATION USING IMAGE PROCESSING TECHNICS.