Bone Fracture Detection using Hand Gesture Recognition

Bone Fracture Detection using Hand Gesture Recognition

Binash Shikah

Computer Dept line

M.H Saboo Siddik Polythenic Mumbai, India

Dhanshree Pansare

Computer Dept line

M.H Saboo Siddik Polythenic Mumbai, India

Munira Ansari

Computer Dept line

M.H Saboo Siddik Polythenic Mumbai, India

Sajid Khan

Computer Dept line

1. Saboo Siddik Polythenic Mumbai, India

   Abstract The bone fracture is a common problem in human beings occurs due to high pressure is applied on bone or simple accident and also due to osteoporosis and bone cancer. Therefor the accurate diagnosis of bone fracture is important aspects in medical field. In this work hand gesture is used for bone fracture analysis. The aim of this project is to develop a Bone Fracture Detection Using Image Processing and Hand Gesture Recognition based efficient system for a quick and accurate classification of bone fractures based on the information gained from the hand gesture and X-ray image. X- ray image of the fractured bone are obtained from hospital and processing techniques like pre-processing, segmentation, edge detection and feature extraction methods are adopted. The processed images will be further classified into fractured and nonfractured bone and compare the accuracy of different methods. This project is fully employed Python programming tool for loading image, image processing and user interface development. Results obtained demonstrate the performance of the bone fracture detection system with some limitations and good accuracy of 85%.

   KeywordsBone fracture detection, image processing and hand gesture.

   1. INTRODUCTION

      Bones are the solid organs in the human body protecting many important organs such as brain, heart, lungs and other internal organs. The human body has 206 bones with various shapes, size and structures. The largest bones are the femur bones, and the smallest bones are the auditory ossicles. Bone fracture is a common problem in human beings. Bone fractures can occur due to accident or any other case in which high pressure is applied on the bones. There are different types of bone fracture occurs are oblique, compound, comminuted, spiral, greenstick and transverse. There are different types of medical imaging tools are available to detecting different types of abnormalities such as X-ray, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), ultrasound etc. X-rays and CT are most frequently used in fracture diagnosis because it is the fastest and easiest way for the doctors to study the injuries of bones and joints. Doctors usually uses x-ray images to determine whether a fracture exists, and the location of the fracture. The

      database is DICOM images. In modern hospitals, medical images are stored in the standard DICOM (Digital Imaging and Communications in Medicine) format which includes text into the images. Any attempt to retrieve and display these images must go through PACS (Picture Archives and Communication System) hardware.

      So, for that quick and accurate diagnosis crucial to the success of any prescribed treatment. Depending upon the human experts alone for such a critical matter have cause intolerance errors. Hence the thought of automatic identification procedure has perpetually been associate degree appealing one the main goal of the paper is to detect the bone fracture from X-ray images and Live hand gesturerecognition.

   2. EXISTING SYSTEM

      1. X-ray image processsing

         First,Preprocessing These stages consist of the procedures that enhance the features of an input X- ray image. Then Edge Detection It is based on analyzing the changes in the intensity in the image. Image segmentation is the fundamental step to analyses image and extract data from them. Then image classifier after classify we detect the fracture.

      2. Hand gesture

         In hand gesture recognition method contain three major stage. The first stage is the object detection. The second stage is object recognition. The third stage is to analyse sequential gestures to identify users instructs and behavior.Depend on this behavior we can identify that there is a fracture or not.

   3. PROBLEM STATEMENT

      In recent years, bone fracture detection and classification has been a widely discussed topic and many researchers have proposed different methods to tackle this problem. Human body suffers from various problems it consists of different parts such as legs, hands, bones, bones get cracked or discontinuity most of the times due to pressure applied on it which may be due

      to the accident, sports while playing etc. Osteoporosis is one of the major problems occurs due to extra use of bones radiologist suggests the patients take x-ray images of the bones for diagnosis purpose. This study is a tutorial review on medical imaging processing and repository techniques appeared in the literature. Many times, it is difficult and time-consuming to find out the location of fracture in the patient who is suffering from pain. The bone fracture is a common problem in human beings occurs due to high pressure is applied on bone or simple accident and also due to osteoporosis and bone cancer. Therefore the accurate diagnosis of bone fracture is important aspects in medical field. In this work X- ray/CT images are used for bone fractureanalysis. A bone fracture is a medical condition where the continuity ojf the bone is broken. A significant percentage of bone fractures occur because of high force impact or stress. Bones are the solid organs in the human body protecting many important organs such as brain, heart, lungs and other internal organs. The human body has 206 bones with various shapes, size and structures. The largest bones are the femur bones, and the smallest bones are the auditory ossicles. Bone fracture is a common problem in human beings. Bone fractures canoccur due to accident or any other case in which high pressure is applied on the bones. There are different types of bone fractureoccurs are oblique, compound, comminuted, spiral, greenstick and transverse.

      Causes of bone fractures can include:

      • Traumatic incidents such as sporting injuries, vehicleaccidents and falls

      • Conditions such as osteoporosis and some types of cancer that cause bones to fracture more easily, meaning even minor trauma and falls can become serious.

   4. PRPOSED SYTEM

      A Bone fracture detection using hand gesture recognition system is developed. The proposed system consists of two main phases, first is image processing and second is hand gesture. In the image processing phase, the images are processed using techniques such as Hear Wavelet transforms, as well as SIFT as feature extractor. These techniques are utilized to enhance the quality of the images and to extract the fracture part of the bone, then its show that there is a fracture. And here our first phase is complete. In hand gesture recognition method contain three major stage. The first stage is the object detection. The second stage is object recognition. The third stage is to analyse sequential gestures to identify users instructs and behavior. Depend on thisbehavior we can identify that there is a fracture or not.

   5. LITERATURE REVIEW

      In this section the related and the limitations of the proposed system will be discussed. Here the works is about hand fracture detection using X-ray image and hand gesture. There are different types of medical imaging tools are available to detecting different types of

      problems as X-ray, Computed Tomography (CT) and Magnetic Resonance Imagig (MRI) etc. But for bone fracture detection X-ray and CT images are most frequently used. Generally doctors prefer the X-ray images to detect the fracture and where it is. Though there are few limitations in X-ray but for low cost, high speed and usability it is very effective. The bone fracture is very natural in old and child ages. The normal and abnormal images have been found and it has been introduced in. In the GLCM (Gray Level Co- occurrence Matrix) approach has been used to segment the x-ray images of the hand and divides those separate bands. After dividing the K-means clustering is used for GLCM texture analysis. As the same way the GLCM approach has been used in where it detects if the fracture in femur exists or not. Here in the preprocessing steps the images has been convert it into binary images and after applying edge detection technique the GLCM based methods used for feature extraction and perform classification. Another segmentation approach used for x- images of hands in as bottom-up region merging method and also compute combinations between local, regional, global and hierarchical distances. In the authors proposed an adaptive interface system called AdAgen that collaborates with trained agent. Here used neural network to detect fracture in long bones and must be mentioned that their simulation result show that how NN perform detection of fracture in leg radiograph. The most effective comparison of x-ray image segmentation techniques has been introduced in. The techniques are thresholding, region-based methods, edge-based segmentation methods, clustering or cluster analysis, classification- based segmentation techniques, level set methods, Active contour models, Active shape models, and Wavelet based techniques and knowledge-based techniques. Among them thresholding, edge detection, classification- based techniques can solve simple image segmentation problems but for complex active contour models and active shape models can be used. On bone fracture detection there has a work where fracture has been detected on measuring the neck shaft angle of the femur. On top of that the authors suggest to use Gabor, markov random field and gradient intensity features and feed them into SVM (Support Vector Machine). They also show that these three features improve the accuracyof

      the model. The system used in described how the carpal bones can be extracted using automatic segmentation methods. In two processes has been used for determining skeletal age. One is image preprocessing using diffusion filter and another is image segmentation using region

      level. Another approaches discussed in where authors proposed to compute the joint width in the x-ray images of hands. In proposed a fusion classification technique for detecting the fracture in tibia bone in x- rayimages.

   6. METHODOLOGY

      The X-ray/CT images are obtained from the hospital that contains normal as well as fractured bones images. In the

      first step, applying preprocessing techniques such as RGB to grayscale conversion and enhance them by using filtering algorithm to remove the noise from the image. Then it detects the edges in images using edge detection methods and segmented the image. After segmentation, it converts each image into a set of features by using some feature extraction technique. Then we build the classification algorithm based on extracted features. Finally, the performance and accuracy of the proposed system are evaluated. The flow diagram of proposed system for detecting the bone fracture in X-ray/CT images.

      The overview of the hand gesture recognition system consists of the following stages. The first stage is the hand gesture image capture stage where the images are taken using digital camera under different conditions such as scaling, translation and rotation. The second stage is a pre-processor stage in which edge detection, smoothing, and other filtering processesoccur. In the next stage, the features of the images of hand gesture are extracted using two methods, namely, hand contour and complex moments. The last stage is the classification using Artificial Neural Network (ANN), where the recognition rate is calculated for both hand contour-based

      ANN and complex moments-based ANN and comparison is carried out. The following is a description of these stages.

   7. BLOCK DIAGRAM

   8. SYSTEM ARCHITECTURE

      First user have to select that what he want to do like imageprocessing or hand gesture.

      If user select image processing then:

      Step 1: Preprocessing These stages consist of the procedures that enhance the features of an input X- rayimage so that he result image improves the

      performance of the subsequent stages of the proposed system.

      Step 2: Edge Detection It is based on analyzing the changes in the intensity in the image. However, the quality of edge detection is highly dependent on lighting conditions, the presence of objects of similar intensities, density of edges in the scene and noise. There are different algorithms for edge detections such as Canny, Laplacian and Sobel. In our experiments, the best results were obtained by using a modified version of the Canny edge detection algorithm in which the contrast is enhanced using a histogram equalizationstep. This finding is in accordance with the Nadernejad et al. result shows the results of using different edge detection algorithms.

      Step 3: Segmentation Image segmentation is the fundamental step to analyses image and extract data from them. It is an operation of partitioning an image into a collection of connected sets of pixels. The main purpose of interest in an image which helps in an image which helps in annotation of the object scene. There are three main approaches of image segmentation which are region approach,boundary approach, and edge approach.

      Step 4: Image classifier In this step different classifier is used like SVM (Support Vector Machine), K-Nearest Neighbor (KNN), Back Propagation Neural Network (BPNN), Nave Byes(NB).

      Step 5: fracture detection The last stage of this system is fracture detection it is performed by the procedures. First, the useful features extracted from the image. And then, these features are used to detect fracture or non-fracture image.

      If user select hand gesture:

      Hand gestures recognition system has been applied for different applications on different domains, as mentioned in including; sign language translation, virtual environments, smart surveillance, robot control, medical systems etc. overview of some hand gesture application areas are listed below.

      Sign Language Recognition: Since the sign language is used for interpreting and explanations of a certain subject during the conversation, it has received special attention. A lot of systems have been proposed to recognize gestures using different types of sign languages. For example recognized American Sign Language ASL using boundary histogram, MLP neural network and dynamic programming matching. recognized Japanese sign language JSL using Recurrent Neural Network, 42 alphabet and 10 words. recognized Arabic Sign language ArSL using two different types of Neural Network, Partially and Fully Recurrent neuralNetwork.

      1. Robot Control: Controlling the robot using gestures considered as one of the interesting applications in this field. proposed a system that uses the numbering to count the five fingers for controlling a robot using hand pose signs. The orders are given to the robot to perform a particular task, where each sign has a specific meaning and represents different function for example, one means move forward, five means stop, and so on.

      2. Graphic Editor Control: Graphic editor control system requires the hand gesture to be tracked and located as a preprocessing operation. used 12 dynamic gestures for drawing and editing graphic system. Shapes for drawing are; triangle, rectangular, circle, arc, horizontal and vertical line for drawing, and commands for editing graphic system are; copy, delete, move, swap, undo, nd close.

      3. Virtual Environments (VEs): One of the popular applications in gesture recognition system is virtual environments VEs, especially for communication media systems. provided 3D pointing gesture recognition for natural human computer Interaction HCI in a real-time from binocular views. The proposed system is accurate and independent of user characteristics and environmental changes.

      4. Numbers Recognition: Another recent application of hand gesture is recognizing numbers. proposed an automatic system that could isolate and recognize a meaningful gesture from hand motion of Arabic numbers from 0 to 9 in a real timesystem using HMM.

      5. Television Control: Hand postures and gestures are used for controlling the Television device. In a set of hand gesture are used to control the TV activities, such as turning the TV on and off, increasing and decreasing the volume, muting the sound, and changing the channel using open and close hand.

      6. 3D Modeling To build 3D modeling, a determination of hand shapes are needed to create, built and view 3D shape of the hand. Some systems built the 2D and 3D objects using hand silhouette. 3D hand modeling can be used for this purpose also which still a promising field of research.

   9. CONCLUSION

The dataset consists of 98 x-ray images; half of them are for normal hand bones and the other half contains a fracturein one of the hands 19 bones. Experiments to evaluate the accuracy of the proposed system are conducted using Weka [11], one of the most commonly-used tools in the machine learning field. Going through the literature, one can see that there are some classifiers work better for problems such as the one at hand.

The fracture in femur bones is very common and is rapidly increasing case in many of the countries. In context of medical science, it is very important to have accurate detection of minor fracture in a bone. So in this approach the classification of fractured and un-fractured bone is done. For which image processing techniques is used which involve preprocessing, feature extraction and classification. The median and average filter is used to get the noise free image and then Logarithmic operator is used for image Enhancement whose factor is evaluated empirically. The morphological operation is performed to processed image and then edge detection technique is used (Sobel filter) to distinguish edges from the image in feature extraction. The sobel edge detection technique gives clear fractured edge but the canny edge detection detects bone edge accurately but

not fractured edge. Then the processed image is given as input for classification using SVM where the image is classified into fractured or un-fractured.

REFERENCES

1. C. M. A. K. Zeelan Basha, Maruthi Padmaja, and G. N. Balaji , Computer Aided Fracture Detection System, 2018 IEEE, 2018.

2. K.Sudha Rani, K.Mani Kumari, Ganti Amulya, Eswar pothineni, velagapudi Pavani, P.Susvitha Leg Bone Fracture Segmentation and Detection using Advanced Morphological Techniques, 2019 IEEE,2019.

3. Md. Badrul Alam Miah and Mohammad Abu Yousuf Detection of Lung Cancer from CT Image Using Image Processing and Neural Network, 2015 IEE,2015.

4. Leonardo Tanzi , Enrico Vezzetti , Rodrigo Moreno and Sandro Moos X-Ray Bone Fracture Classification Using Deep Learning: A Baseline for Designing a Reliable Approach,2020, IEEE,2020

1. [5] Vikas Kajla, Anju Gupta and Anil Khatak Analysis of X-Ray Images with Image Processing Techniques: A Review 2018, International Conference on Computing Communication and Automation(ICCCA),2018.

2. [6] Jayesh S. Sonkusare M.E, Nilkanth B. Chopade, Ravindra Sor ARDE and Sunil L. TadeM.E Hand Gesture Recognition System, 2015 IEEE, 2015.

1. Mohammad Badrul Alam Miah and Afroza Akter Detecting and Determining the Types of Hand Bone Fracture Using K-Means Clustering,2018 ISSN, 2018.

2. ZakriaQadir, Muhammad Ali, T. Nesimoglu Design and Development of a Low Cost Device for Bone Fracture Detection Using FFT Technique on MATLAB, 2018 IEEE department of Electrical and Electronics Engineering, 2018.

3. Shanthi Raju Lanka and Srinivas Yarramalle A METHODOLOGY FOR IDENTIFYING THE FRACTURE DETECTION USING STATISTICAL MIXTURE MODELS, 2018 International Journal of Mechanical Engineering and Technology (IJMET),2018

4. Akshay Trimbak Chikhalekar Akshay Trimbak Chikhalekar, 2016 International Research Journal of Engineering and Technology (IRJET), 2016.

5. Ms. Kalpana Lamb, Prof.Mrs. Swati Madhe Hand gesture recognition based bed position control for disabled patients, 2016 Conference on Advances in Signal Processing (CASP), 2016.

6. Ankur Mani Tripathi, Abhishek Upadhyay, Akanksha Singh Rajput, Ajay Pal Singh and Brajesh Kumar Automatic detection of fracture in femur bones using image processing, 2017 InternationaConference on Innovations in Information, Embedded and Communication Systems (ICIIECS), 2017.

7. Anu T C, Mallikarjunaswamy M.S, Rajesh Raman. Detection of Bone Fracture using Image Processing Methods, 2015 National Conference on Power Systems & Industrial Automation (NCPSIA), 2015.

8. Trupti Patil, Suresh C Kuri, Sagar S Santagi A Surway on fracture detection in leg bone using x-ray image 2020 International Research Journal of Engineering and Technology (IRJET), 2020.

9. Mariam M Saii , Ali M Mayya Hand and Finger Fracture Detection and Type Classification Using Image Processing, 2018 International Journal of Computer Science Trends and Technology (IJCST), 2018.

10. O. Bandyopadhyaya, A. Biswasb, and B. B. Bhattacharyaa Classification of Long-Bone Fractures based on Digital- Geometric Analysis of X-ray images, 2016 IEEE,2016.

11. B. Gajjar, S. Patel, A.Vaghela Fracture detection in X-ray images of long bone 2017 E-ISSN: 2347-2693,2017.

12. KamilDimililer Intelligent bone fracture detection system2017 IEEE2017.

13. Avinash Vishnu V, Divya Prakash J, Swathika R, Sree Sharmila

    T. Detection and classification of long bone fractures2015 IEEE, 2015.

14. Anu T C, Mallikarjunaswamy M.S. and Rajesh Raman Detection of Bone Fracture using Image Processing Methods, 2015 IEEE,2015.