Neural Network Approach To Mathematical Expression Recognition System

Neural Network Approach To Mathematical Expression Recognition System

Surendra P. Ramteke Dhanashri V. Patil

M. E. Scholar

Nilima P. Patil

Department of E&TC Engineering, SSBTs College of Engineering & Technology, Jalgaon. (M.S.), INDIA

Abstract Now a days there is a lot of development in smart devices which are combination of human intelligence and machines. Character recognition is one of the example of smart device and Mathematical expression recognition is belongs to such device which is developed to recognize handwritten as well as printed mathematical symbols and expressions. This system converts scientific or engineering documents in to its electronic form. In this work a purely offline approach is utilized. The main phases carried out to achieve the goal are pre-processing, segmentation, feature extraction, classification and recognition of symbols as well as mathematical expression. Out of these phases mainly important are feature extraction and classification because it affects overall accuracy of the system. In this system centroid and bounding box are the main features that are extracted from each character and this system is accomplished using neural network approach for the recognition of the expressions and symbols which provides 90% recognition rate.

Key Words mathematical expression recognition, segmentation, feature extraction, classification.

1. INTRODUCTION

   There is a wealth of mathematical knowledge that could be potentially very useful in many computational applications. Mathematical expression recognition is one of the research topics from several decades but it is still an area of research topic because there are lots of challenges in this system. It is very important regarding scientific document image analysis and this system have applications like scientific document digitization, information retrieval or accessibility for blind people [14]. The input for this system is mathematical expressions and symbols. The input of mathematical expression into computers is very difficult than that of plain text because mathematical expression contains special symbols, Latin or Greek letters and different operators with digits [6].

   For on-line approach system utilizes the temporal information about strokes input. Where off-line recognition deals with the image representation of mathematical expressions, which may be printed or handwritten. This system faces lots of difficulties because if we consider handwritten mathematical expression then there are variations occur in the size, font of symbol, writing style varies person to person and quality of image also matters.

   Figure.1 shows the system Overview for mathematical expression recognition [15]. The first step carried out is data acquisition. This data is then acquired from optical

   scanner. Next step is pre-processing image cleaning takes place. Along with that image is converted to form of suitable subsequent processing like size normalization, skeletonization and noise removal takes place. After pre- processing image is applied to segmentation and feature extraction. In next step the document is segmented in sub components and separating of each character is takes place. After segmentation the feature set which is useful for training of the system and recognition is extracted.

   Figure 1: Overview of Mathematical Expression Recognition System.

   During classification, a character is placed in the appropriate class to which it belongs. In recognition step training and testing are apply.

2. STEPS FOR RECOGNITION

   In this section the important processes that are carried out in the mathematical expression recognition are discuss briefly [4]. At first expression is written on a white plain paper with use of black color marker pen. The program designed for this system is able to read the expression may available in the file format of .jpg, .bmp or .gif.

   1. Data Collection and Pre-processing

      At first data is collected from different persons and a database is created [18]. In this system we have considered 0 to 9 numbers, some alphabetical characters and symbols as well as special characters. All the numbers, symbols and characters are putted in sequential form. They are isolated with certain distance from each other and a fixed sequence has been maintained throughout in rows and column. The database is as shown in fig. 2 as follows. Here we have considered handwritten as well as printed set of characters.

      Figure 2: sample of database.

      On the input image to the system first pre-processing is carried out. We have to normalize the expression, resize it to proper dimensions, and remove the background noise. The raw data is subjected to different processes under pre- processing so that it can be applied to further. It essentially enhances the image makes it suitable for segmentation. This produces data which is easy for character recognition of system so it can operate accurately. The main functions carried out under this step are as follows.

      1. Read input image.

      2. Convert RGB image into gray scale image.

      3. Convert gray scale into black and white image.

      4. Remove component whose size less than 50 pixels.

      5. Invert and reshape image.

         First the input image is read. This is generally available in color format so it is converted into gray scale image first. RGB to gray conversion takes place with certain values 0.2989, 0.5870 and 0.1140 respectively for RGB. And this gray scale image is then converted in black and white image with the threshold value of 0.5. To perform certain morphological operations it is necessary to carried out this conversion in to binary image. Image cleaning is takes place to remove unwanted spots from the image. The component whose size is less than 50 pixels is removed from the image. After that image inversion and reshaping takes place the result for this process is as shown.

         Figure 3: input to the preprocessing

         Figure 4: output of preprocessing

   2. Segmentation

      In the segmentation, the input image is segmented into individual characters and then, each character is resized into m x n pixels towards the training network. In the proposed system, the pre-processed input image is segmented into isolated characters by assigning a number to each character using a labeling process. In the segmentation stage, an image of sequence of characters is decomposed into sub-images of individual character [10]. This labeling provides information about number of characters in the image. In the labeling operation two values are find out first is total number of objects

      connected in the image and second is to each and every component present in the image an integer value greater than 0 is assigned.

3. FEATURE EXTRACTION

   After pre-processing and segmentation step feature set is extracted this is further used for training and recognition step. Feature sets play one of the most important roles in a recognition system. A good feature set should represent characteristic of a class that helps distinguish it from other classes, while remaining invariant to characteristic differences within the class.

   1. After labeling region properties are find out from the labeled component.

   2. Rectangle is drawn with respect to all number of components, bounding box, position, and edge color, and radius and line width.

   3. Properties compute the features that are area, bounding box and centroid.

   4. Calculate centroid with respect to two diections horizontally as well as vertically and store it in two variables column wise.

   Area finds out number of pixels in the region. Centroid means to find out weights of component and it locates the centre of component. For these numbers of dots are first find out and then average value is taken. Center is calculated of each and every component. The first element of Centroid is the horizontal coordinate or x-coordinate of the center of mass, and the second element is the vertical coordinate or y-coordinate. Fig shows the bounding box and centroid of the image.

   Figure 5: Location of centroid and bounding box

   As shown in figure the centroid is indicated with blue color * indicate in each component and a bounding box is drawn with red color box. And values of centroid in intefer form for the same expression is as follow.

   x = 66.3574 118.5674 120.3139 212.2624 296.2003

   374.8640 440.1329

   y = 65.7469 72.6738 56.9854 53.5622 57.5554

   51.5280 50.4820

   Here in the expression there are 7 components are present so seven x-value of centroid and seven y-value of centroid

   are evaluated. In this expression equal to (=) component is treated as two separate components. Means upper horizontal line and lower horizontal line both are treated as different components.

   The smallest rectangle containing the region are bounding box. Those objects which are not present in bounding boxes are the unwanted components and these are discarded. Width to height ratio (WH) of bounding box can be expressed as:

   this method error is propagated back to the hidden unit and training of back propagation network is done in following way as shown in training algorithm [12, 20].

   Training Algorithm

   1. Initialization of weights.

   2. Feed forward of input training patterns.

   3. Calculation of back propagation error.

   4. Updating weights and biases.

   =

   The relative height ratio (RH) and relative width ratio (RW) also required for the bounding box [15].

4. CLASSIFICATION

   A Neural Network is defined as a computing architecture that consists of massively parallel interconnection of simple neural processors. It can perform computations at a higher rate compared to the classical techniques because of its parallel nature [4]. ANN is inspired by the way biological nervous system such as the brain process information. The key element of this paradigm is the novel structure of the information processing system. And it is composed of large number of highly interconnected processing elements (neurons). ANN is configured for specific application, such as data classification.

   When ANN processed data then there are two main

   The neurons present in the hidden and output layer have biases, which are connections from unit whose activation is always one. The inputs apply to BPN and output from it is always either binary or bipolar.

   Initialization of weights

   Step 1: Initialize weights and biases.

   Step 2: Until stopping condition is false repeat step 3 to10. Step 3: For each training pair do steps 3 to 9.

   Feed forward of input training patterns

   Step 4: Each input unit receives input signal and transfer it to the next layer which is nothing but hidden layer.

   Step 5: each hidden unit sums it weighted input signals to calculate net input which is given by

   stages that are training stage and classification stage. In classification stage samples are passed as input to the

   = +

   (1)

   ANN, resulting an output representing what ANN believes to be the most correct output. To be a successful classification it must be preceded by a training stage [12]

   Calculate the output of hidden unit by applying its activation function over . = ( )

   The output signal from hidden layer is transferred to output layer.

   Step 6: For each output unit , calculate net input.

   = +

   (2)

   Apply activation function to compute the output signal which is given by = ( ).

   Figure 6: Example of Feed Forward Network.

   A Back propagation network consists of three layers of unit consists of input layers, hidden layer and output layer. These units are connected in feed forward fashion means input units are fully connected with hidden layer units and hidden units are connected with output units. Gradient descent method is used for the weight update algorithm. In

   Calculation of back propagation error

   Step 7: Each output unit receives a target pattern corresponding to input training pattern and compute error correction term.

   = ( ) (3)

   On the basis of basis of calculated error correction term, update changes in weight and biases. = and

   = . It sends to hidden layer backward.

   Calculation of back propagation error

   Step 8: Each hidden unit sums its delta input from output unit:

   Table 1: Network parameter value

   =1

   =

   Parameters	Value
   Transfer Function	logsig
   Network training function	Gradient descent
   Learning Rule	Momentum
   network performance function	Mean squared error (MSE)
   Learning constant	0.01
   Goal	0.001
   Number of Epochs	5000
   Number of iteration after which output have shown	20
   Momentum Term	0.95

   (4)

   The term gets multiplied with derivative of( )to calculate the error term.

   = ( ) (5)

   On the basis of calculated , update the change in weights and biases: = and =

   Updating weights and biases

   Step 9: Each output unit updates the bias and weights:

   = + (6)

   = + (7) Each hidden unit updates its bias and weights:

   = + (8)

   = + (9)

   Step 10: Check for the stopping condition. The stopping condition may be a certain number of epochs reached [12, 20].

   Network parameters:

   Table 1 shows the parameters that are set for network training and its respective values also listed [17].

   Results:

   Figure 7: Output window for the Mathematical Expression Recognition System.

   Here in this approach we have used GUI, so in output window we have created two axes, two buttons that are read expression and recognize. It consists of two text boxes. Her we have read handwritten expression first and on second axis each and every component is recognized and on output text box recognized expression is generated in printed form.

   Author	Method	Recognition rate
   Taik Heon Rhee, Jin Hyung Kim[8]	Structural Analysis using hypothesis generation	89.0%
   N. Venkateshrao, G. Rama Mohan Babu[12]	Artificial Neural Network	95.6%
   Francisco Álvaro, Joan Andreu Sánchez[3]	Weighted nearest neighbor, Hidden Markov models (HMM), k- Nearest Neighbor	98.5%
   Utpal Garain, B.	K-NN Classifier	90.42%
   B. Chaudhuri[5]
   	Hidden Markov	92.19%
   	Model
   Ahmad Montaser	multi-layer
   Awal, Harold	perceptions	87.5%
   Mouchère,	neural network
   Christian Viard-	(MLP)
   Gaudin[2]
   Our Approach	Neural Network	90%

   1. Shrikrishna,

   2. Ravindrababu,

   Table 2: Comparison of different Classifiers

   Table 2 shows various methods used by different authors for symbol and mathematical expression recognition system. This comparison gives idea about recognition rate of the different systems.

   The above table shows the different methods for mathematical expression recognitions and has recognition rate details respectively. So among these methods it is concluded that neural network has good recognition rate 90%. And it is more advantageous due to its Adaptive learning, Self-Organization, Real Time Operation, Fault Tolerance via Redundant Information Coding.

5. CONCLUSION

   Recognition of mathematical expression consists main steps pre-processing, segmentation, feature extraction and Classification. In this technique to obtain maximum recognition rate is very difficult due to the ambiguities of

   mathematical expressions. This paper provides an neural network approach which gives 90% recognition rate.

6. REFERENCES

1. B. Q. Huang and M-T Kechadi, A Structural analysis approach for online handwritten mathematical expressions, ijcsns international journal of computer science and network security, vol.7 no.7, july 2007.

2. Ahmad-Montaser Awal, Harold Mouchère, Christian Viard-Gaudin, A Hybrid classifier for handwritten mathematical expression recognition, IRCCyN/IVC- UMR CNRS 6597.Rue Christian Pauc-BP 50609-44306 Nantes CEDEX-France.

3. Francisco Alvaro, Joan Andreu Sánchez, Comparing several techniques for offline recognition of printed mathematical symbols, 2010 international conference on pattern recognition.

4. Nafiz arica, an off-line character recognition system for free style handwriting thesis submitted on sep 1998.

5. Utpal garain, B. B. chaudhuri, Recognition of online handwritten mathematical expressions, IEEE transactions on systems, man, and cyberneticspart b: cybernetics, vol. 34, no. 6, december 2004.

6. Sanjay S. Gharde, Baviskar Pallavi V., K. P. Adhiya,

   Evaluation of classification and feature extraction techniques for simple mathematical equations, international journal of applied information systems (ijais)

   issn : 2249-0868 foundation of computer science fcs, new york, usa volume 1 no.5, February 2012.

7. Francisco Alvaro Munoz, Offline recognition of printed mathematical expressions using stochastic context- free grammars, iarfid master thesis universidad politecnica de valencia september 10, 2010

8. Taik Heon Rhee, Jin Hyung Kim, Efficient search strategy in structural analysis for handwritten mathematical expression recognition, Published in Pattern Recognition Journal Volume 42 Issue 12,December 2009,Pages 3192-3201

9. J.pradeep, E. Srinivasan and S. Himavathi, Diagonal based feature extraction for handwritten alphabets recognition, international journal of computer science & information technology (ijcsit), vol 3, no 1, feb 2011.

10. Kam Fai Chan, Dit Yan Yeung, Department of Computer Science, The Hong Kong University of Science

    and TechnologyMathematical expression recognition survey(2000), international journal on document analysis.

11. Qi Xiangwei Pan Weimin Yusup Wang Yang, The study of structure analysis strategy in handwritten recognition of general mathematical expression, 2009 international forum on information technology and applications.

12. N. Venkteswara Rao, Dr. A. Shrikrishna, Dr. B. Ravindrababu, G.Rrama Mohan Babu, An efficient feature extraction and classification of handwritten digits using neural networks", ijcsea vol.1,no.5, october 2011.

13. Ping Zhang, Reliable recognition of handwritten digits using a cascade ensemble classifier system and hybrid features, a thesis in the department of computer science and software engineering, concordia university montreal, quebec, canada april 2006.

14. Francisco a´ lvaro,Joan-andreu sa´nchez, Jose´ miguel bened, Recognition of printed mathematical expressions using two-dimensional stochastic context-free grammars, 2011 international conference on document analysis and recognition.

15. Anshul gupta, Manisha srivastava, Offline handwritten character recognition, a thesis submitted in iit, guwahati,April,2011

16. Velappa ganapathy, and Kok leong liew,

    Handwritten character recognition using multiscale neural network training technique, world academy of science, engineering and technology 15-2008.

17. Amit choudhary and Rahul rishi, Improving the character recognition efficiency of feed forward bp neural network, international journal of computer science & information technology (ijcsit), vol 3, no 1, feb 2011

18. S. V. Rajashekararadhya, Dr P. Vanaja ranjan,

    Efficient zone based feature extration algorithm for handwritten numeral recognition of four popular south indian scripts, journal of theoretical and applied information technology 2005

19. Srinivasa kumar devireddy, Settipalli appa rao, Hand written character recognition using back propagation network, journal of theoretical and applied information technology 2005 – 2009

20. S jayaraman, S Esakkirajan, T Veerakumar, Digital image processing, Tata McGraw- hill Education,2011- 0-07-014479-8 ISBN(13):978