I Recognise U Raspberry Pi based Wearable Face Recognition Device with Image Tagging Facility for Blind

I Recognise U Raspberry Pi based Wearable Face Recognition Device with Image Tagging Facility for Blind

Chaithra .S*, Prakruthi .N*, Vadiraja Acharya*,

*Information Science Department, JyothyInstitute of Technology, Bangalore, India

AbstractIn this paper, we present a wearable device to assist visually impaired users in social interactions(I Recognise You). One of the most challenging tasks for people who are blind or have low vision is to recognize people in a variety of social interactions. Recognition through voice is commonly used to identify people. However, this may be difficult or unfeasible in some circumstances. The proposed device is a wearable Face Recognition device which allows blind and low vision people to interact with society. Using face recognition and OpenCV technology, the device will identify classmates, relatives and colleagues by giving a message on a earphone to notify the user and has a facility to tag image by giving some identity or name for new people. If the same person appears infront of the user then the name of a person is given as a message on a earphone. The proposed device supports to recognize people easily and its societal benefit is to improve the access, integration and independence of the blind or visually impaired individuals in workspace or educational settings.

Keywords Face Recognition; Raspberry Pi; Open CV; Social Interaction,Visual impairment;

1. INTRODUCTION

   One of the most challenging tasks for people who are blind or have a low vision or prosopagnosia disorder is to recognize people in a variety of social interactions. Recognition through voice is commonly used to identify people. However , this may be difficult or unfeasible in some circumstances.

   Face recognition is the primary means by which humans identify each other. Visually impaired people and some people with prosopagnosia[6] may only have difficulty of recognizing a familiar face; others will be unable to discriminate between unknown faces, while still others may not even be able to distinguish a face as being different from an object and cannot perceive faces, which means they are unable to rapidly identify people around them.This puts them at a major disadvantage in social interactions.This project proposes to develop a wearable device which allows blind and low vision people to interact with society. Using face recognition technology, the device will identify classmates, relatives and colleagues by giving some identity or name for new people. If the same person appears infront of the user then the name of a person is given as message on aearphone.The face is our primary

   focus of attention in social life playing an important role in conveying identity and emotions.

   We can recognize a number of faces learned throughout our life span identity faces at glance even after years of separation. This skill is quite robust despite of large variations in visual stimulus due to changing condition , again and distractions such as beard , glasses, or changes in hairstyles. In this chapter it supports to recognise people easily and its societal benefit is to improve the access , integration and independence of the blind or visually impaired in workplace or educational settings.

   Computational models of face recognition are interesting because they can contribute not only to theoretical knowledge but also to practical applications. The major challenges of the Proposed technique include (a) Face Detection (b) Training the new face images (c) Face recognition, under various conditions like lighting, with or without spectacles (d) database (e)voice input& output (f) tagging facility.

   Face detection is used on many places nowadays especially the websites hosting images like Photobucket and Facebook . The automatically tagging feature adds a new dimension to the database. We motivates the practical utility and academic interest of a system that learns its wearers social from the perspectives of assistive technology and wearable computing.

2. WEARABLE DEVICE

   1. Hardware

      1. Raspberry Pi Model, 512 Mb with a nice black plastic case :The Raspberry Pi [12] is a low cost, credit-card sized computer that plugs into a computer monitor or TV, and uses a standard keyboard and mouse. It has the ability to interact with the outside world, and has been used in real time applications.

      2. Membrane keypad: Membrane keypad is a simple keypad with 13 buttons (3 x 4) and since every key has its own wire line, no matrix code is required – just treat these like every day switches. The membrane is soft and has a removable paper backing to expose a strong adhesive so you can stick this on an enclosure and feed the cable through a slot. It include a 7-pin extra-long header strip so you can connect this to gpio pins of

         Raspberry Pi with ease. The user controls the device using this keypad. Most blind users can learn to operate it efficiently.

      3. USB earphone:It is the main requirement for user to record or to receive audio message.The user can enroll the name of new face image to the database by recording an audio message through a earphone and for the identified person, enrolled name is sent as a voice message on earphone to notify the user.

      4. Camera module: The Raspberry Pi camera module [12] can be used to take high-definition video, as well as stills photographs. It attaches via a 15cm ribbon cable to the CSI port on the Raspberry Pi. It is very popular in Face recognition Application. A camera captures 5 images for training and 10 images for recognition.

      5. SDHC card: The SD card is a key part of the Raspberry Pi, provides the initial storage for the Operating System and files. Storage can be extended through many types of USB connected peripherals. 16 GB SD card is used for our Project.

      6. Power Bank: Raspberry Pi can work without the main connection by using power bank that can be connected to Pi using an USB cable.

   2. Software

      1. Install Raspbian:Raspbian software packages [12] is install to run Raspberry Pi and enable camera.

      2. Install CMake: Thecross-platform, open-source build system. CMake is a family of tools designed to build, test and package software. CMake[12] is used to control the software compilation process using simple platform and compiler independent configuration files.Cmake is mandatory for compile.

      3. Install OpenCV Lib for face recognition: OpenCV[10] is released under a BSD license and hence its free for both academic and commercial use. It has C++, C, Python and Java interfaces and supports Windows, Linux, Mac OS, iOS and Android. OpenCV was designed for computational efficiency and with a strong focus on real-time applications. Written in optimized C/C++, the library can take advantage of multi- core processing. Enabled with OpenCL, it can take advantage of the hardware acceleration of the underlying heterogeneous compute platform. Adopted all around the world, OpenCV has more than 47 thousand people of user community and estimated number of downloads exceeding 9 million. Usage ranges from interactive art, to mines inspection, stitching maps on the web or through advanced robotics.

      4. Install OpenCV 2.3:Install both dev lib and python lib. Our soft is C-written.

      5. Install face recognition API:libfacerec is a library for face recognition in OpenCV. It has been merged into OpenCV 2.4 (contrib module) and

         both implementations are synchronized. So if you are in (a recent) OpenCV 2.4: There is no need to compile libfacerec yourself, you have everything toget started.

      6. Install gstream: Install gtream for audio input and output.

      7. Install sqlite3: Install sqlite to create database for storing data.

   1. WORKING

      The Face and Interaction Models represent the models used for facial and social interaction detection, respectively. In practice, each are trained once, prior to system use, and rarely updated. The Acquaintance Model, in its current form, is simply a face recognition model, and is updated frequently. While acquaintances are recognized in real-time per the green workflow, the current work performs Interaction Detection, and the subsequent user-labeling of audio subsamples for the regular updating of the Acquaintance Model.

      We set out to design a wearable FR device that can learn and assist with a (potentially blind) wearers social interactions. Its intended utility is to unobtrusively inform the wearer about the presence of nearby acquaintances.

      Two scenarios we specifically design for are:

      1. when the wearer is walking between classes/offices, and identifying them in time for the wearer to initiate a social interaction.

      2. Identifying proximate acquaintances that have just approached or already engaged the wearer so that the wearer can respond with the appropriate level of familiarity.

      Each scenario requires that the device be able to accurately distinguish acquaintances from non-acquaintances. The device is able to communicate information to the wearer in a timely and private manner in the form of a full name . Notification of full names is accomplished by broadcasting an audio message over earphone. Identification of acquaintances is handled by face detection and recognition.

      Figure1: Workflow Diagram

      Figure1: Workflow Diagram

      1. Face Detection

         The original API (application protocol interface) provided in IRU device allows us to capture images. Firstly, OpenCV face detector is applied to find faces in the incoming images. It is very robust in finding frontal faces, as it is trained with a huge number of training samples . The OpenCV detector can detect the faces(frontal faces) in the poses of looking at the camera of IRU device using the algorithm based on Haar features. Face detector which is trained by Haar Cascade Classifier are applied to find faces in the input image frame. Detected face image is captured, automatically cropped and resized to 222 x 222 pixels. Captured images are used for training database.

      2. Training

         In training stage, for each person enrolled in the database, five face images are collected. All these face images undergoes image transformation using various image filtering techniques such as changing brightness-constrast, titling of an image. After image transformation the number of images increases from 5 to 25 face images. Trained images are stored in the database automatically , path of each image is updated in the file called Comma Separated Value (CSV) file and the number of the people in the database is updated in the file called category file.At each training stage , CSV and category file is updated. When more people have to be enrolled in the database, the incoming face images are transformed using the above mentioned image filtering techniques and then the transformed face images are stored in database.

         1. Brightness-Constrast:An image must have the proper brightness and contrast for easy viewing. Brightness refers to the overall lightness or darkness of the image. Contrast is the difference in brightness between objects or regions. Brightness and constrast of an image can be changed by changing the value of alpha and beta. Figure shows the effect of changing britness and constrast of an image by giving different values for alpha and beta.

            Figure2: Effect of changing britnesss-constrast

            Figure2: Effect of changing britnesss-constrast

         2. Tilt: A detected face image is tilted to different angles.

            Figure3 : Image tilted to right and left

            Figure3 : Image tilted to right and left

            Figure4: Image of a database images

            Figure4: Image of a database images

      3. Face Recognition

      Face detection is the first step of face recognition [2] as it automatically detects a face from a complex background to which the face recognition algorithm can be applied. But detection itself involves many complexities such as background, poses, illumination etc. Some of the face recognition algorithms used are Principal Component Analysis(PCA), Multilinear Principal Component Analysis (MPCA) and Linear Discriminant Analysis(LDA), Eigen faces , Fisher faces ..etc.

      Every algorithm has its own advantage. Fisher face algorithm is most popularly used for Face Recognition Projects. While PCA is the most simple and fast algorithm, MPCA and LDA which have been applied together as a single algorithm named MPCALDA provide better results under complex circumstances like face position, luminance variation etc.

      Steps involved in Face Recognition :

      • The camera constantly captures images ("frames")

      • When the camera has captured a frame, OpenCV face detection algorithm is run to find a face on the image. If no face is found the frame isdiscarded, if a face is found the rectangle containing the face is "cut out"of the image and passed on to the Fisherface recognition algorithm.

      • Fisherfacealgorithm identify the face. They either report that the face isunknown (it belongs to a person who is not enrolled into IRU database), or they report the name of the enrolled person the face belongs to.

      • In order to identify a person, IRU iterates over all the framescaptured and processed within the last

        2 seconds. For every candidate in this list the procedure shows thescore of confidence level of each candidate when identified on a frame.

      • The candidate who has the lowest confidence score is returned as the identified person ("unknown" is considered as a candidate with highest confidence score).

      • Identified person name is notified to a user through a voice message over a earphone.

        The Fisherfacesalgorithm[12] we are used basically goes like this:

      • Construct the ImagematrixX with each column representing an image. Each image is a assigned to a class in the corresponding class vector C.

      • Project X into the (N-c)-dimensional subspace as P with the rotation matrix WPca identified by a Principal Component Analysis, where

        • N is the number of samples in X

        • c is unique number of classes (length(unique(C)))

      • Calculate the between-classes scatter of the projection P as Sb = \sum_{i=1}^{c}

        N_i*(mean_i – mean)*(mean_i – mean)^T, where

        • mean is the total mean of P

        • mean_i is the mean of class i in P

        • N_i is the number of samples for class i

      • Calculate the within-classes scatter of P as Sw =

        \sum_{i=1}^{c} \sum_{x_k \in X_i} (x_k – mean_i) * (x_k – mean_i)^T, where

        • X_i are the samples of class i

        • x_k is a sample of X_i

        • mean_i is the mean of class i in P

      • Apply a standard Linear Discriminant Analysis and maximize the ratio of the determinant of between-class scatter and within-class scatter. The solution is given by the set of generalized eigenvectors Wfld of Sb and Sw corresponding to their eigenvalue. The rank of Sb is atmost(c-1), so there are only (c-1) non-zero eigenvalues, cut off the rest.

      • Finally obtain the Fisherfaces by W = WPca * Wfld.

   2. RESULTS

The proposed device randomly captures face images for training and then automatically updates CSV and Category file. In the case of training, Person name is enrolled into database by a voice message and in the other case for recognition, using trained images,recognizes a particular person by comparing or matching images in the database by playing an voice message of a particular person.

Figure5: Training the new face.

Figure6: Recognizing the trainined face.

CONCLUSION

This paper helps visually impaired and blind individuals to identify persons in front of them and makes them independent to live in society. Our proposed device works in two modes of operations: Face Detection and training and Face Recognition. We propose a device that includes face detection, regularized methods for training and testing of individuals in an unconstrained environment.

ACKNOWLEDGEMENT

We would like to thank Mr. Vadiraja Acharya, Assistant Professor in the department of ISE at Jyothy Institute of Technology, who support us in development of this device

. We are also thankful to the visually impaired users and there valuable information in this work.

REFERENCES

1. Krishna, S., Little, G., Black, J., Panchanathan, S.: A wearable face recognitionsystem for individuals with visual impairments. In: ACM SIGACCESS Conf. onComputer and Accessbility. (2005) 106113

2. Utsumi, Y., Kato, Y., Kunze, K., Iwamura, M., Kise, K.: Who are you?: A wearableface recognition system to support human memory. In: ACM Proceedings of the4th Augmented Human International Conference. (2013) 15015314Mandal et al.

3. Singletary, B.A., Starner, T.E.: Symbiotic interfaces for wearable face recognition.In: In HCII2001 Workshop On Wearable Computing. (2001)

4. H.F., E., Haan, D., Campbell, R.: A fifteen year follow-up of a case of developmental prosopagnosia. Cortex, a Journal Devoted to the Study of the Nervous Systemand Behaviour 27 (1991) 489509

5. Bate, S.: Face recognition and its disorders. UK: Palgrave Macmillan (2013)

6. Kennerknecht, I., Grueter, T., Welling, B., Wentzek, S., Horst, J., Edwards, S.,Grueter, M.: First report of prevalence of non- syndromic hereditary prosopagnosia(hpa). American Journal of Medical Genetics 140 (2006) 16171622

7. Grter, T., Grter, M., Carbon, C.: Neural and genetic foundations of face recognitionand prosopagnosia. Jounal of Neuropsychol. 2 (2008) 7997

8. Wang, X., Zhao, X., Prakash, V., Shi, W., Gnawali, O.: Computerized-eyewearbased face recognition system for improving social lives of prosopagnosics. Proceedings of the 7th International Conference on Pervasive Computing Technologiesfor Healthcare (2013) 7780

9. Nametag. http://www.nametag.ws/ (2014)

10. OpenCV: Open source computer vision. http://opencv.org/ (2014)

11. Raspberry Pi : https://thinkrpi.wordpress.com

12. http://www.bytefish.de/blog/fisherfaces/