Sign Language Recognition using Feature Selection Based on Binary Black-winged Kite Optimization Algorithm

Sign Language Recognition using Feature Selection Based on Binary Black-winged Kite Optimization Algorithm

Jixi Gu

Nanjing No.1 Middle School International Department Nanjing, China

AbstractSign language recognition aims to help understand sign language and bridge the communication gap between speech or hearing-impaired and other non-impaired individuals. Sign language recognition belongs to the area of pattern recognition. Feature selection is an important basic approach for solving the pattern recognition problem, and it reduces and refines the feature set of data so that the generated feature subset can further improve the pattern recognition accuracy. In this paper, a new binary black-winged kit optimization algorithm-based feature selection method, BBKAFS, is proposed. When applying to sign language recognition, BBKAFS can not only reduce the feature dimensions, but also obtain higher recognition accuracies on three public sign language data sets.

Keywordssign language recognition; feature selection; swarm intelligence; pattern recognition; black-winged kit algorithm

1. INTRODUCTION

   Sign language is different from oral language, and it is a sophisticated manual language employing hand gestures, body movements, and facial expressions to express individual feelings, thoughts, intentions, needs, and so on[1]. Therefore, sign language becomes a dominant tool of connection between speech or hearing-impaired communities and the general people. The understanding of sign language routinely depends on visual inputs, and automatic sign language recognition can use pattern recognition techniques to realize the translation of this visual-gestural language into the normal spoken or written language[2]. The higher the sign language recognition accuracy is, the smoother the communication between deaf and dumb people and the social world will be.

   In the era of big data and artificial intelligence, pattern recognition algorithms or models require to have the faster learning speed, the stronger generalization capability, the better recognition accuracy, and so on. Feature Selection, which is often regarded as one of basic techniques related to the area of pattern recognition, can take advantage of some evaluation criteria to create the new feature subset based on original data. When recognition algorithms or models are applied to the new generated feature subset of data, the better performance will be achieved. Feature selection methods are broadly categorized into three kinds: embedded methods, wrapper methods, and filter methods[3]. Wrapper methods refine the feature subset by the operational results of one classification algorithm[4], and in this paper, the proposed approach based on binary swarm intelligence optimization is classified as this type.

   Swarm intelligence optimization is a class of optimization methods, and it can search for the global solutions to problems by mimicking and modelling the behavioral characteristics of one group, and its algorithms not only are employed flexibly, but also are easy to be understood. Popular swarm intelligence optimization algorithms include particle swarm optimization[5], grey wolf optimization[6], whale optimization algorithm[7], Harris Hawk Optimization(HHO)[8], Black Kite optimization Algorithm (BKA)[9], and so on. It is worth noting that BKA is a latest meta-heuristic optimization algorithm inspired by the migration and attack behaviors of black-winged kites, and it exhibits the satisfactory performance in complex functions and engineering cases. Since feature selection is a binary discrete problem in fact, at present, many binary swarm intelligence optimization algorithms are commonly used in the wrapper based feature selection method, such as binary particle swarm optimization[10], binary grey wolf optimization[11] and Binary HHO(BHHO)[12]. Like BBHO based Feature Selection (BBHOFS) in [12], these binary versions of methods consider the classification error rate of a learning model as the optimization objective, and can help to search for and obtain better feature subsets in a discrete space.

   In this paper, first we design a Binary BKA(BBKA), which employs the V-type function to transform continuous space into discrete space, and next present a new BBKA based Feature Selection method(BBKAFS), and finally apply this proposed BBKAFS to sign language recognition. Experimental results on three public sign language data sets demonstrate that when compared to other approaches, BBKAFS can not only find a feature subset with smaller size, but also acquire higher recognition accuracies.

2. BINARY BLACK-WINGED KITE ALGORITHM (BBKA)

   BBKA based on the currently presented BKA[9] that mimics and models the predatory and migration behavior of black- winged kites mainly consists of three stages as follows:

   1. Initialization Stage

      , xij , , xid

      Assume that X is a pack of n black-winged kites, and let X x1, x2, , xi , , xn where xi is the position of the ith black-winged kite and each xi xi1 , xi 2

      where xij represents the jth dimension of the position of the ith

      black-winged kite and d is the number of dimensions of the position space. Each xij can be initialized as follows:

      , and the curve image of this V-shaped function is shown in

      Fig.1. Furthermore, note that r1 takes the different value for different i and different j .

      xij rand int(0,1)

      (1)

      Last, after the completion of the continuous to binary

      where rand int(0,1) generates a random number of 0 or 1. Assume that f is the optimization object of BBKA, that is,

      conversion,

      function. If

      xi and

      x

      b

      i

      f x

      f

      b should be evaluated by the fitness

      x

      i

      , then let x xb ; otherwise x

      the fitness function.

      f xi is used to evaluate the position of

      i i i i

      remains unchanged.

      the ith black-winged kite, and a kite located at the best position

      xbest

      is considered as the leader of kites. The position xbest is

      defined as:

      xbest arg min( f (xi ))

      xi , xi X

      (2)

      , and

      f (xbest ) is the optimal value of the fitness function.

   2. Attacking Stage

      When hunting small prey in the air, the black-winged kite needs to frequently adjust its position. First, to mimic this behavior, a mathematical model of updating the position xi of the ith black-winged kite is defined as:

      c xi (sin( ) 1) xi

      xi

      x

      i

      xi (2 1) xi

      (3)

      where

      c is the new generated position of the ith black-

      Fig.1. The curve image of the V-type function

      winged kite in the attacking stage, like the original BKA, is a random number between 0 and 1, 0.9 , and is gotten

      as follows:

      q 2

   3. Migration Stage

   First, like BKA, a hypothetical migration behavior of the black-winged kites is mimicked and a related mathematical model is defined as:

   2 Q

   x w (x x ) f (x ) f (x )

   0.05 e

   (4)

   xc

   i i best i g

   (7)

   i x w (x h x ) f (x ) f (x )

   where q is the current iteration number, and Q is the i best i i g

   maximum number of iterations. Moreover, note that takes the different value for different i .

   Second, BBKA asks to search the optimal position in the 0 1 binary space, but all new positions generated by Eq.(3) are in the continuous space. Therefore, a continuous to binary map is needed to fulfil the space conversion, and it is defined as:

   x r1 V (xc )

   where c is the new generated position of the ith black-

   x

   i

   winged kite in the migration stage, xbest is the position of the black-winged kites leader and is currently optimal, g is a random integer with the range of 1, n , and w and h can be respectively computed as:

   xb

   ij ij

   (5)

   h 2sin( )

   (8)

   ij ij

   ij x r1 V (xc )

   2

   and

   where

   xij

   the jth dimension of the position xi

   that is not

   w tan((r2 0.5) )

   (9)

   updated by Eq.(3), xc is the jth dimension of the positions xc where takes the same value as of Eq.(3), w is a Cauchy

   ij

   in the continuous space,

   i

   x

   ij

   b is the jth dimension of the

   random number related to Cauchy distribution, and r2 is a random number between 0 and 1. Besides, note that g takes

   positions

   xb in the binary space, x

   is the complement of

   the different values for different i , and

   r2 also takes the

   i ij

   2

   xij , r1 is a random number between 0 and 1, V () is a V- type transfer function defined as:

   distinct values for distinct i .

   Second, a continuous to binary space conversion is still employed in the migration stage, and a map similar to Eq.(5) is defined as:

   V (t)

   arctan(

   t)

   (6)

   x r3 V (xc )

   2 xb

   ij ij

   (10)

   ij ij

   ij x r3 V (xc )

   i

   where

   xij

   the jth dimension of the position xi

   that is not

   Step 6: For xb , perform KNN and get the classification error

   updated by Eq.(7) after the attacking stage,

   c is the jth

   rate on TrnFS and ValFS , and calculate the fitness

   x

   ij

   dimension of the positions xc in the continuous space, xb is

   function value f xb . If f

   , then let

   i ij

   x

   b

   i

   f x

   the jth dimension of the positions xb in the binary space,

   i i

   x xb ; otherwise x remains unchanged.

   i i i i

   xij is the complement of xij , r3 is a random number between 0 and 1, and V () is Eq.(6). Additionally, note that

   Step 7: Set the random numbers g , r2 and r3 , update xi and get xc by Eq.(7), and convert xc to xb by Eq.(10).

   i i i

   x

   b

   i

   f x

   i

   r3 takes the different value for different i and different j . Last, after the completion of the continuous to binary

   conversion, x and xb should be also evaluated by the fitness

   Step 8: For xb , perform KNN and get the classification error rate on TrnFS and ValFS , and calculate the fitness

   x

   b

   i

   f x

   i i

   value

   f xb . If

   f

   , then let x

   xb ;

   function. If

   f

   , then let x

   xb ; otherwise x

   i

   i i i

   i

   i i i

   remains unchanged.

   otherwise xi remains unchanged.

3. BBKA-BASED FEATURE SELECTION (BBKAFS)

   Step 9: Set i i 1. If i n , then goto Step 5.

   Step 10: Sort all f xi ( i 1, 2, , n ), and acquire

   xbest

   BBKA-based feature selection, that is, BBKAFS, belongs to the wrapper method, and for it, this paper designs a fitness function f related to the classification error rate obtained

   from a simple classification algorithm, that is, K-Nearest Neighbors (KNN). Assume that each xi ( i {1, 2, , n}) is viewed as a candidate feature subset, and this fitness function f [13] is defined as:

   f x (1 ) (x ) (xi ) (11)

   by Eq.(2) again.

   Step 11: Set q q 1. If q Q , then goto Step 4; otherwise BBKAFS ends, and xbest is the finally attained optimal feature subset.

4. Sign Language Recognition Experiments

   1. Experimental Settings and Data Sets

      First, since the essence of sign language recognition is regarded

      i i

      max

      as the classification problem of sign language, and in order to

      measure the performance of the above proposed BBKAFS, we

      where (xi ) is the classification error rate on the data with the feature subset xi , max is the size of the original whole feature set, (xi ) is the size of the feature subset xi , and

      0.01.

      The detailed procedure for BBKAFS is given as follows:

      Step 1: Let n be the number of candidate feature subsets, and

      Q be the maximum number of iterations. Set q 1 ,

      0.9 , and 0.01.

      Step 2: Let TrnFS and ValFS respectively be the training and validation data specially used for feature selection,

      X be a set of all feature subsets, and each xi ( i {1, 2, , n} ) in X be a candidate feature subset.

      Step 3: Initialize each xi by Eq.(1). For each xi , carry out classification of KNN and obtain the classification error rate (xi ) on TrnFS and ValFS , and calculate the fitness function value f xi by Eq.(11). Sort all f xi ( i 1, 2, , n ), and acquire xbest by Eq.(2).

      Step 4: Set i 1.

      Step 5: Set the random numbers and r1, update xi and get

      xc by Eq.(3), and convert xc to xb by Eq.(5).

      employed KNN for the data classification and the combination with the wrapper-based feature selection, and we also compared BBKAFS-based KNN with BBHOFS-based KNN and KNN without feature selection.

      Second, the parameter K of KNN classifier was set to 5, For BBHOFS and BBKAFS, the number n of candidate feature

      subsets was set to 10, the maximum number Q of iterations was set to 100, Eq.(11) was used as the fitness function, and other parameters of BBHOFS was set according to [12].

      Next, three sign language data sets from Kaggle[14] were used

      in experiments. There three data sets were as follows: British Sign Numbers(BSN), German Sign Language Alphabet(GSLA) and Leap Motion American Sign Language(LMASL). Each data set was divided into the training and testing sets with the approximate ratio of 8:2. KNN, BBHOFS-based KNN and BBKAFS-based KNN were performed and evaluated on these training and testing data. Moreover, for each data set, this training data that accounts for 80% of the total data was again split into TrnFS and ValFS in the approximate ratio of 5:5, and TrnFS and ValFS were specially used for the feature selection of BBHOFS and BBKAFS. Table I lists main characteristics of three sign language data sets, and Table II gives the number of samples in the testing data, the training data, TrnFS and ValFS for each data set.

      Finally, experiments were done by Matlab on MacOS running on the personal computer with Inter Core i7 and 16GB

      i i i

      table i. Sign Language Data Sets Used in

      Experiments

      Data set	The number of samples	The size of the feature set	The number of classes
      BSN	1194	63	11
      GSLA	7306	63	24
      LMASL	5145	428	18

      2058

      Data set	Testing data	Training data	TrnFS	ValFS
      BSN	239	955	477	478
      GSLA	1461	5845	2922	2923
      LMASL	1029	4116	2058

      TABLE II. THE NUMBER OF SAMPLES IN THE TESTING, TRAINING, TRNFS AND VALFS DATA

      table iii. Comparison of The Sizes of Optimal Feature Subsets

      Data set	No feature selection	BBHOFS	BBKAFS
      BSN	63	30	7
      GSLA	63	32	10
      LMASL	428	207	29

      Fig.2. Comparison of classification accuracies on BSN

      Fig.3. Comparison of classification accuracies on GSLA

      Fig.4. Comparison of classification accuracies on LMASL

      RAM. All experimental results were averaged on 20 runs for each data set.

   2. Experimenal Results

   First, the comparison of the sizes of optimal feature subsets generated by BBHOFS and BBKAFS is reported in Table III. As shown in Table III, on the one hand, when no feature selection is employed, the optimal feature subset is the original entire feature set, and on the other hand, BBKAFS can acquire the lower size of the optimal feature subset than BBHOFS on each sign language data set.

   Second, the classification accuracies on three sign language data sets are respectively shown in Fig.2, Fig.3 and Fig.4. In Fig.2 and Fig.3, over 90% classification accuracies are attained only by BBKAFS-based KNN, and in Fig.4, although there is little different classification performance between BBHOFS- based KNN and BBKAFS-based KNN, the latter, that is, the proposed BBKAFS-based KNN, still can achieve the highest accuracy. Furthermore, Fig.2, Fig.3 and Fig.4 describe that when compared with feature selection based KNN, such as BBHOFS-based and BBKAFS-based KNN, the simple KNN without feature selection underperforms on three sign language data sets.

5. CONCLUSIONS

To improve the accuracy of sign language recognition, this paper proposes an effective black-winged kite optimization algorithm based feature selection method, that is, BBKAFS. Compared to other approaches for sign language recognition, our method (i) enables to refine and get the optimal feature subset with the smaller size, and (ii) has the better sign language recognition performance.

REFERENCES

1. S. Alyami, H. Luqman, and M. Hammoudeh, Reviewing 25 years of continuous sign language recognition research: advances, challenges, and prospects, Information Processing & Management, vol.61, No.5, pp.103774, September 2024.

2. Y.Q. Zhang, X.W. Jiang, Recent advances on deep learning for sign language recognition, Computer Modeling in Engineering & Sciences, vol.139, no.3, pp.2399-2450, March 2024.

3. A. Moslemi, A tutorial-based survey on feature selection: recent advancements on feature selection, Engineering Applications of Artificial Intelligence, vol.126, part D, pp.107136, November 2023.

4. B.H. Nguyen, B. Xue, and M.J. Zhang, A survey on swarm intelligence approaches to feature selection in data mining,Swarm and Evolutionary Computation, vol.54, pp.100663, May 2020.

5. A.G. Gad, Particle swarm optimization algorithm and its applications: a systematic review, Archives of Computational Methods in Engineering, vol.29, pp.2531-2561, April 2022.

6. S. Mirjalili, S.M. Mirjalili, and A. Lewis, Grey wolf optimizer,

   Advances in Engineering Software, vol.69, pp.46-61, March 2014.

7. S. Mirjalili, A. Lewis, The whale optimization algorithm, Advances in

   Engineering Software, vol.95, pp.51-67, May 2016.

8. A.A. Heidari, S.Mirjalili, H. Faris, and et al., Harris hawks optimization: algorithm and applications, Future Generation Computer Systems, vol.97, pp.849-872, August 2019.

9. J. Wang, W.C. Wang, X.X. Hu, and et al., Black-winged kite algorithm: a nature-inspired meta-heuristic for solving benchmark functions and engineering problems, Artifical Intelligence Rewiew, vol.57, March 2024.

10. J.W. Too, A.R. Abdullah, and N.M. Saad, A new co-evolution binary particle swarm optimization with multiple inertia weight strategy for feature selection, Informatics, vol.6, no.2, May 2019.

11. J.W. Too, A.R. Abdullah, N.M. Saad, and et al., A new competitive binary grey wolf optimizer to solve the feature selection problem in EMG signals classification, Computers, vol.7, no.4, November 2018.

12. J.W. Too, A.R. Abdullah, and N.M. Saad, A new quadratic binary harris hawk optimization for feature selection, Electronics, vol.8, no.10, October 2019.

13. T. Dokeroglu, A. Deniz, H.E. Kiziloz, A comprehensive survey on recent metaheuristics for feature selection, Neurocomputing, vol.494, no.14, pp.269-296, July 2022.

14. Kaggle datasets website, https://www.kaggle.com/datasets.