- Open Access
- Total Downloads : 243
- Authors : S. S. Shanbhag, G. R. Udupi, K. M. Patil, K. Ranganath
- Paper ID : IJERTV3IS20719
- Volume & Issue : Volume 03, Issue 02 (February 2014)
- Published (First Online): 24-02-2014
- ISSN (Online) : 2278-0181
- Publisher Name : IJERT
- License: This work is licensed under a Creative Commons Attribution 4.0 International License
Probability Neural Network Classification Model of Brain Tissue Pathologies Using Signal Intensity Gradient Technique
S. S. Shanbhag1, G. R. Udupi1, K. M. Patil2, K. Ranganatp
1Electronics and Communication Engineering, Gogte Institute of Technology, Belgaum, India
2Retired, Indian Institute of Technology (Madras), Presently at Belgaum, India
3RAGAVS, Diagnostics and Research Center Pvt. Ltd., Bangalore, India
Abstract –Automated classification in medical images is motivated by the need of high accuracy when dealing with a human life. Conventional operator-assisted classification techniques are not viable for large amounts of medical data and are generally non-reproducible. In the proposed work, we have made an effort to realize an automated classification system to derive useful information about the brain pathologies, namely, cerebral infarction, Intracerebral Haemorrhage (ICH) and brain tumors (glioma and meningioma), by using prior information about the signal intensity characteristics on Diffusion Weighted-Magnetic Resonance (DW-MR) images. The automated system is designed using Probability Neural Network (PNN) architecture that works on the quantified signal intensity variations on DW- MR images, via Signal Intensity Gradient (SIG) parameter. The brain pathology PNN models were able to accurately (100%) categorize ICH subjects into their respective stages, and presented an overall efficiency of 86.67% in classifying the infarct subjects. Also the model presented an overall efficiency of 92.86% in differentiating the subjects with glioma and meningioma. The PNN models could aid in better understanding the degree of brain pathology in the absence of an expert, directly in terms of the brain image signal intensity parameters, and additionally aid in identifying the extent of damage caused to the brain tissues.
Keywords-Diffusion weighted images, Glioma, Infarction, Intracerebral haemorrhage, Magnetic resonance imaging, Meningioma, Signal intensity, Signal intensity gradient
-
INTRODUCTION
Neurological diseases are common important disorders resulting in various degrees of disability and loss of productive life [1]. Early and accurate diagnosis of the brain pathologies plays a vital role in the implementation of successful treatment and thereby improving the quality of life. In recent times, when neurological disorders are gradually increasing, we are also witnessing remarkable advances in brain imaging, which is becoming an increasingly essential tool in clinical research and care [1]. Conventional imaging techniques of monitoring and diagnosing brain pathologies rely on detecting the presence of particular features, by a human observer. Such human
assisted diagnostic techniques are not viable for large amounts of medical data and are generally non-reproducible [2]. Thereby several techniques for automated diagnostic systems have been developed in recent years, which could improve the results of humans, so that false negative cases are kept at a very low rate [2]. In this domain the applications of neural networks to feature classification have been extensively studied in the past many years. Various kinds of neural network architectures including MultiLayer Perceptron (MLP) neural network, Radial Basis Function (RBF) neural network, Self-Organizing Map (SOM) neural network and Probability Neural Network (PNN) have been proposed [3]. However, because of ease of training and a sound statistical foundation in Bayesian estimation theory, PNN has become an effective tool for solving many classification problems [4-6]. PNN often learns more quickly than many neural network models, and has had success on a variety of applications [7].
Diffusion Weighted – Magnetic Resonance Imaging (DW-MRI) is one of the most rapidly evolving imaging techniques, both for very basic clinical needs and for advanced diagnosis and treatment planning [8]. It is an extremely sensitive tool to monitor brain water diffusion changes with high temporal and spatial resolution, and therefore allows for more detailed insight into the pathophysiology of the brain pathologies [9]. Diffusion Weighted – Magnetic Resonance (DW-MR) images present variation in the signal intensity characteristics relative to the amount of restriction to water molecules, post the occurrence of the brain pathologies [10]. Therefore, it is very essential in understanding the characteristic features of the signal intensity variations on DW-MR images post brain pathologies, as it may allow for the assessment of the brain affected areas and thereby allow for the diagnosis and proper planning of the clinical treatment. In the present study, PNN architecture is employed to develop a classification model based on the quantified signal intensity variations on DW-MR images, derived from the subjects with brain pathologies, using Signal Intensity Gradient (SIG) parameter. Studies are performed on a set of DW-MR images of the human brain (in the axial plane as shown in
Fig.1) to understand the characteristics of the signal intensity variations, for the subjects with brain pathologies, namely, cerebral infarction, Intracerebral Haemorrhage (ICH) and brain tumor (glioma and meningioma); the details of the study given in our earlier published works [11, 12]. An effort is made to grade the signal intensity variations on DW-MR images using SIG values, for the subjects with these brain pathologies, compared to the corresponding SIG values obtained from the respective contralateral normal hemispheres. The quantified signal intensity variations using SIG values are employed in developing brain pathology PNN models, to automatically classify the stages of cerebral infarction [13], categorize the stages of ICH [14], and differentiate between the types of brain tumors, namely, glioma and meningioma.
Fig. 1 DW-MR image showing region-of-interest on pathology (glioma) side and on contralateral side (axial slice 13)
The PNN models developed in the present work could positively be employed to derive valuable information about the particular brain tissue pathology, which may assist the medical personnel in the speedy diagnosis and execution of treatment. Further, the proposed computer based technique could simplify the estimation process and provide information essential for the further management and therapeutic decisions, even in the absence of a medical expert.
-
CLINICAL DATA AND DIFFUSION WEIGHTED IMAGING
-
Subjects
The clinical data in the present study is obtained from
RAGAVS Diagnostic and Research Center, Bangalore, India, and Vikram Hospital, Bangalore, India. The ethics approval is obtained from the committee of clinical research at the RAGAVS Diagnostic and Research Center, and Vikram Hospital, to carry out the investigations on the clinical data provided. Applying the clinical inclusion criteria, we identified 98 cases with clinically definite cerebral infarctions with known symptom onset, which were employed in training and testing the PNN model. This population consisted of 64 male and 34 female subjects, ranging in age from 25 to 95 years (mean, 62.87 years). DW-MRI examinations were performed on 24 subjects within 24 hours of symptom onset; 15 subjects between days 1 and 4; 38 subjects between days 5 and 9; 12 subjects
between days 10 and 14; and 09 subjects after day 15.
Among all the ICH subjects admitted, we retrospectively selected 42 cases that showed isolated ICH without the
presence of underlying tumor or infarction on initial radiologic and follow-up examinations, which were employed in training and testing the PNN model.This population consisted of 29 male and 13 female subjects, ranging in age from 28 to 85 years (mean, 58.62 years). DW-MRI examinations were performed on 09 subjects within 24 hours of symptom onset; 05 subjects between days 1 and 7; 15 subjects between days 7 and 14; and 13 subjects after 14 days.
Findings of DW-MRI examinations performed on 22 cases (15 male, 07 female; mean age, 56.55 years; age range, 39 83 years) with clinically proved glioma and 12 cases (04 male, 08 female; mean age, 56.50 years; age range, 26 81 years) with clinically proved meningioma, were also retrospectively selected, and employed in training and testing the PNN model.
-
Imaging parameters
All the subjects underwent clinical MR imaging with 1.5 Tesla symphony maestro class MR scanning system from Siemens. DW-MRI was performed by using a multisection, single-shot, spin-echo, echo-planar pulse sequence with following parameters: Repetition Time [TR] = 3200 ms, Echo Time [TE] = 94 ms, acquisition matrix = 128 x 128, Field of View [FOV] = 230 mm x 230 mm, and diffusion gradient value of b = 1000 s/m2 along 19 axial sections, 5 mm thick section and intersection gap of 1.5 mm.
-
-
METHODOLOGY
-
Signal intensity gradient
In biological tissues, the presence of various tissue components restrict the Brownian motion in certain directions, and this may render the water diffusion have different values when measured in different directions. Thereby, the molecular mobility may not be the same in all directions, resulting in a spatial direction-related diffusion property termed as diffusion anisotropy [15]. In order to take care of the orientation dependent contrast (diffusion anisotropy), the SIG value in the present work is evaluated using the compass detection technique, that allows extracting explicit information about the change in intensity in any direction.
We have employed Robinson compass operator comprising of 8 kernels, as shown in Fig. 2, to calculate the SIG value. Robinson compass operator is chosen as it is easier to implement compared to the other compass detectors, because it relies only on coefficients of 0, 1 and 2, and is symmetrical about their directional axis (axis with zeros). The set of 8 kernels applied in Robinsons compass operator are produced by taking one kernel and rotating its coefficients circularly. These kernels are convolved with the input image to obtain the SIGs along the corresponding directions. GS, GSE, GE, GNE, GN, GNW, GW, and GSW, represents gradients along South, South-East, East, North- East, North, North-West, West and South-West directions respectively. The magnitude of the maximum change in the SIG is the maximum value gained from applying all the 8 kernels to the pixel neighbourhood [16, 17]. For each pixel, the local gradient magnitude is estimated with the maximum
response of all the masks at this pixel location, as given in Eq. 1 [16].
SIGmax = max{GS, GSE, GE, GNE, GN, GNW, GW, GSW} (1)
Fig. 2 Robinson kernels and their orientation
The SIG value evaluated in the area of brain pathology is compared to the corresponding SIG value on the contralateral normal hemisphere of the same subject to obtain the relative signal intensity SIG (RSIG) value for the subject, as given in Eq. 2.
SIG value on p-athology side
RSIG correspond ing SIG value on contralateral side (2)
correspond ing SIG value on contralateral side
Fig. 3 PNN network architecture [19]
(Note: The symbols and notations are adopted as used by MATLAB Neural Network Toolbox. Dimensions of arrays are marked under their names)
The PNN has three layers: the Input Layer, Radial Basis Layer and the Competitive Layer. It is assumed that there are Q input vector/target vector pairs, with each target vector having K elements. Each input vector is associated with one of K classes. The dimension of the input vector, denoted as p, is R x 1. In radial basis layer, the vector distances between input vector and the weight vector made of each row of weight matrix, W, are calculated. The weights of the radial basis layer, IW1, 1, are set to the transpose of the matrix formed from the Q training pairs, P'. When an input is presented, the ||dist|| box produces a vector whose elements indicate how close the input is to the vectors of the training set. These elements are multiplied, element by element, by the bias and sent to the radial basis (radbas) transfer function. The transfer function for a radial basis neuron is given by Eq. 3 and the plot of the radial basis transfer function [19] is shown in Fig. 4.
The RSIG value is quantified across different subjects diagnosed with the brain pathology, and is employed in differentiating pathology tissues from healthy tissues, and further used in the categorization of the different stages of the pathology (cerebral infarction/ICH) and also in differentiating between the different pathology types (glioma and meningioma). The details of the calculation of SIG and RSIG values are elaborated in our earlier studies [11, 12].
-
Probability neural network
All PNNs predominantly classifiers, are a special form of RBF network that can map any input pattern to a number of classifications. A function is radial basis if its output
radbas (n) en2
Fig. 4 Radial basis function
(3)
depends on the distance of the input from a given stored vector. The RBF network [18] is a special class of multilayer feedforward networks wherein each unit in the hidden layer employs a RBF, such as a Gaussian kernel, as the activation function. The RBF is centered at the point specified by the weight vector associated with the unit. Both the positions and the widths of these kernels must be learned from training patterns. Each output unit implements a linear combination of these RBFs. We have chosen a basic Matlab PNN for its simple structure and training manner. The network architecture of a PNN employed in the present work is shown in Fig. 3.
The biases are all set to 0.8326/spread, where, spread is the spread value of RBF. As spread becomes larger, the designed network takes into account several nearby design vectors. An input vector close to a training vector is represented by a number close to 1 in the output vector a1. If an input is close to several training vectors of a single class, it is represented by several elements of a1 that are close to 1 [19].
The weights of the competitive layer, LW2, 1, are set to the matrix of target vectors. Each vector has a 1 only in the row associated with that particular class of input, and 0s elsewhere. The competitive layer sums these contributions for each class of inputs to produce as its net output a vector
of probabilities. Finally, compete transfer function picks the maximum of these probabilities, and produces a 1 for that class and a 0 for the other classes. Thus, the network classifies the input vector into a specific K class because that class has the maximum probability of being correct [19, 20].
-
Quantitative brain pathology models using SIG
Brain pathology models for the subjects with cerebral infarction, ICH and brain tumors (glioma and meningioma) with SIG parameter are developed using PNN architecture as shown in Fig. 3. The PNN is implemented by using MATLAB version 7.7. The data set is divided into two separate data sets the training data set and the testing data set. The training data set is used to train the network, whereas the testing data set is used to verify the accuracy and the effectiveness of the trained network, for the classification process. Spread is an important parameter for PNN, and the accuracy of the classification process may vary with different spread values. The task in designing a PNN thereby lies in selecting spread values for the particular classification problem. To facilitate choose the spread value in the present work we have made use of the
leave one out method of cross-validation. Here cross- valdation is carried out using a single observation from the training sample (N) as the validation data, and the remaining observations (N-1) as the training data. The value of spread is adjusted so that the chosen data for validation from the training set is appropriately classified. The process is repeated, such that each observation in the training sample is used once as the validation data [21]. By doing so we choose the optimal spread value, giving maximum accuracy for the given classification problem. The brain pathology models using PNN architecture developed in the present work could help in identifying the stage of cerebral infarction, stage of ICH, and differentiating between the tumor types, quantitatively, relative to the SIG parameter.
-
-
RESULTS AND DISCUSSIONS
-
Cerebral infarction classification model using SIG
A block schematic diagram of the cerebral infarction classification model using SIG parameter developed using PNN architecture (Fig. 3) is shown in Fig. 5.
Fig. 5 Block schematic diagram of cerebral infarction classification model using SIG
Studies are performed on cerebral infarct subjects to understand the signal intensity variation distribution on DW-MR images, using SIG parameter. The RSIG value is given as the input variable, and the output of the PNN block is the resulting stage of infarction. An attempt is made to associate the RSIG parameter evaluated from the infarct subjects with the different stages of infarction, by designing a PNN and training it with adequate number of input-output patterns. The developed PNN model could help in
classifying the stages of infarction, quantitatively, using SIG measurements.
The PNN model is tested with infarct subjects in different stages of infarction. A total number of 65 data sets (split into different stages of infarction as shown in Table 1) are used for training the PNN. Similarly, a total number of
33 data sets (split into different stages of infarction as shown in Table 1) are used for testing the PNN. The spread value of the RBF is used as a smoothing factor and classifier accuracy is examined with different values of spread. The optimal spread value for the classification of cerebral infarct subjects using SIG parameter is found to be 0.5.
The performance of the classification model is evaluated as the percentage of the total number of patterns in the testing data set that are correctly classified. The number of data sets used for training, testing, and the percentage of correct classification obtained from the PNN model, shown in Fig. 5, is tabulated in Table 1. It is observed from Table 1 that the cerebral infarct classification model using SIG
Table 1 Number of data sets used for training, testing, and the number of correct classification obtained for tested data using cerebral infarct
classification model with SIG
Cerebral infarction stage (days)
Number of data sets used for training
Number of data sets used for testing
Percentage of correct classification
Stage 1
(<1)
16
8
75.00
Stage 2
(1 4)
9
6
83.33
Stage 3
(5 9)
26
12
75.00
Stage 4
(10 14)
8
4
100.00
Stage 5
(>15)
6
3
100.00
parameter is able to classify clearly (100%) the infarct subjects in stage 4 and 5 only. The performance of the model for stage 1, 2, and 3 could be improved by training with more number of corresponding input patterns.
-
ICH classification model using SIG
A block schematic diagram of the ICH classification model using SIG parameter developed using PNN architecture (Fig. 3) is shown in Fig. 6.
Fig. 6 Block schematic diagram of ICH classification model using SIG
Studies are performed on ICH subjects to understand the signal intensity variation distribution on DW-MR images,
using SIG parameter. The RSIG value is given as the input variable, and the output of the PNN block is the resulting stage of ICH. An attempt is made to associate the RSIG parameter evaluated from the ICH subjects with the different stages of ICH, by designing a PNN and training with adequate number of input-output patterns. The developed PNN model could help in classifying the stage of ICH, quantitatively, using SIG measurements.
The PNN model is tested with ICH subjects in different stages of ICH. A total number of 28 data sets (split into different stages of ICH as shown in Table 2) are used for training the PNN. Similarly, a total number of 14 data sets (split into different stages of ICH as shown in Table 2) are used for testing the PNN. The spread value of the RBF is used as a smoothing factor and classifier accuracy is examined with different values of spread. The optimal spread value for the classification of ICH subjects using SIG parameter is found to be 0.7.
The performance of the classification model is evaluated as the percentage of the total number of patterns in the testing data set that are correctly classified. The number of data sets used for training, testing, and the percentage of correct classification obtained from the PNN model, shown in Fig. 6, are tabulated in Table 2. It is observed from Table 2 that the ICH classification model is able to classify clearly (100%) the ICH subjects in all the stages of ICH.
Table 2 Number of data sets used for training, testing, and the number of correct classification obtained for tested data using ICH classification
model with SIG
ICH stage (days)
Number of data sets used for
training
Number of data sets used for
testing
Percentage of correct classification
Hyperacute (< 1)
6
3
100
Acute (1 7)
3
2
100
Late subacute
(7 14)
10
5
100
Chronic
(>14)
9
4
100
-
Brain tumor classification model using SIG
A block schematic diagram of the brain tumor classification model using SIG parameter developed using PNN architecture (Fig. 3) is shown in Fig. 7. Studies are performed on subjects with glioma and meningioma to understand the signal intensity variation distribution on DW-MR images, using SIG parameter.
Fig. 7 Block schematic diagram of brain tumor classification model using SIG
The RSIG value is given as the input variable, and the output of the PNN block is the resulting type of tumor, i.e.
either glioma or meningioma. An attempt is made to associate the RSIG parameter evaluated from the tumor subjects, with the appropriate tumor type, by designing a PNN and training with adequate number of input-output patterns. The developed PNN model could help in the differential diagnosis of brain tumors namely glioma and meningioma, quantitatively, using SIG measurements.
The PNN model is tested for the subjects with gliomas and meningiomas. A total number of 15 data sets with glioma and 08 data sets with meningioma are used for training the PNN. Similarly, a total number of 07 data sets with glioma and 04 data sets with meningioma are used for testing the PNN. The spread value of the RBF is used as a smoothing factor and classifier accuracy is examined with different values of spread. The optimal spread value for the classification of brain tumor subjects using SIG parameter is found to be 0.7.
The performance of the classification model is evaluated as thepercentage of the total number of patterns in the testing data set that are correctly classified. The number of data sets used for training, testing, and the percentage of correct classification obtained from the PNN model, shown in Fig. 7, is tabulated in Table 3.
Table 3 Number of data sets used for training, testing, and the number of correct classification obtained for tested data using brain tumor SIG
classification model
Tumor type
Number of data sets used for
training
Number of data sets used for
testing
Percentage of correct classification
Glioma
15
7
85.71
Meningioma
8
4
100
It is observed from Table 3 that the brain tumor classification model is able to clearly (100%) differentiate the subjects with meningiomas. The performance of the model could be improved by training with more number of corresponding input patterns for subjects with glioma.
-
-
CONCLUSIONS
Quantitative brain pathology models using PNN architecture are developed for the classification of the different stages of cerebral infarction and ICH, and differentiation between the two tumor types, namely, glioma and meningioma. Separate PNN models are developed for each type of pathology using RSIG as the input parameter. The PNN model developed for the classification of cerebral infarct stages is able to classify the infarct cases into their respective stages with an overall efficiency of 86.67%. The PNN model developed for the classification of ICH stages is able to classify the ICH cases accurately (100%) into their respective stages. Also, the PNN model developed for the differential diagnosis of glioma and meningioma is able to differentiate between the two tumor types with an overall
efficiency of 92.86 %. The models can be additionally trained with sufficient number of data sets of signal intensity measurements to further enhance their performance. The PNN models provide helpful information that could positively assist the medical personnel in the speedy diagnosis and execution of treatment of the respective brain pathologies. Thereby, we suggest that the adoption of the proposed method in the clinical diagnosis and treatment of these brain pathologies could be instructive in devising corrective methods and providing proper treatment, to protect the subjects from additional damage to their brain tissue.
ACKNOWLEDGEMENTS
The authors would like to thank Dr. Sudhir Pai (Sr. Consultant, Neurosurgeon, Vikram Hospital, Bangalore, India) for his valuable assistance in data collection and Dr. Deepak Y. S. (Consultant Radiologist, Bangalore, India) for his helpful and constructive suggestions during the planning and development of this research work.
REFERENCES
-
M. Gourie-Devi, Organization of Neurology Services in India: Unmet Needs and the Way Forward, Neurology India, January- March 2008, 56(1), pp. 4-12.
-
M. V. Kumar and S. Kulkarni, Tumors Classification Using PNN Methods, International Journal of Soft Computing and Engineering, November 2012, 2(5), pp. 266-268.
-
K. Z. Mao, K. C. Tan, W. Ser, Probabilistic Neural Network Structure Determination for Pattern Classification, IEEE Transactions on Neural Networks, July 2000, 11(4), pp. 1009-1016.
-
D. F. Specht, Probabilistic Neural Networks, Neural Networks, 1990, 3, pp. 109-118.
-
C. Kramer, B. Mckay, J. Belina, Probabilistic Neural Network Array Architecture for ECG Classification, Proceedings of Annual International Conference of IEEE Engineering in Medicine and Biology Society, 1995, 17, pp. 807-808.
-
M. T. Musavi, K. H. Chan, D. M. Hummels, K. Kalantri, On the Generalization Ability of Neural Network Classifier, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, 16(6), pp. 659-663.
-
I. M. M. El Emary and S. Ramakrishnan, On the Application of Various Probabilistic Neural Networks in Solving Different Pattern Classification Problems, World Applied Sciences Journal, 2008, 4 (6), pp. 772-780.
-
R. Rajeshkannan, S. Moorthy, K. P. Sreekumar, R. Rupa, N. K. Prabhu, Clinical Applications of Diffusion Weighted MR Imaging: A Review, Indian Journal of Radiology and Imaging, 2006, 16(4), pp. 705-710.
-
L. C. Maas and P. Mukherjee, Diffusion MRI: Overview and Clinical Applications in Neuroradiology, Applied Radiology, November 2005, 34(11), pp. 44-60.
-
D. Le Bihan, Looking into the Functional Architecture of the Brain with Diffusion MRI, Nature Reviews. Neuroscience, June 2003, 4(6), pp. 469-480.
-
S. S. Shanbhag, G. R. Udupi, K. M. Patil, K. Ranganath, Quantitative Analysis of Diffusion Weighted MR Images of Intracerebral Haemorrhage by Signal Intensity Gradient Technique, International Journal of Medical Imaging, 2013, 1(1), pp. 12-18.
-
S. S. Shanbhag, G. R. Udupi, K. M. Patil, K. Ranganath, Quantitative Analysis of Brain Diffusion Weighted MR Images for Infarct Using Signal Intensity Gradient Technique, International Journal of Biotechnology, 2013, 111, pp. 215-223.
-
J. M. Shen, X. W. Xia, W. G. Kang, J. J. Yuan, L. Sheng, The Use of MRI Apparent Diffusion Coefficient (ADC) in Monitoring the Development of Brain Infarction, BMC Medical Imaging, 2011, 11(2).
-
W. G. Bradley Jr., MR Appearance of Hemorrhage in the Brain, Radiology, October 1993, 189(1), pp. 15-26.
-
J. V. Hajnal, M. Doran, A. S. Hall, A. G. Collins, A. Oatridge, J. M. Pennock, I. R. Young, G. M. Bydder, MR Imaging of Anisotropically Restricted Diffusion of Water in the Nervous System: Technical, Anatomic, and Pathologic Considerations, Journal of Computer Assisted Tomography, January 1991,15(1), pp. 1-18.
-
K. C. Sudeep, J. A. Majumdar, Novel Architecture for Real Time Implementation of Edge Detectors on FPGA, International Journal of Computer Science Issues, 2011, 8(1), pp. 193-202.
-
G. S. Robinson, Edge Detection by Compass Gradient Masks, Computer Graphics and Image Processing, 6(5), October 1977, pp. 492-501.
-
S. Haykin, Neural Networks: A Comprehensive Foundation, 1994, MacMillan College Publishing Co., New York.
-
MATLAB Users Guide, The Math Works Inc., Natick, MA 01760-2098, 1998.
-
P. D. Wasserman, Advanced Methods in Neural Computing, 1993, Van Nostrand Reinhold, New York, pp. 35-55.
-
A. M. Molinaro, R. Simon, R. M. Pfeiffer, Prediction Error Estimation: a Comparison of Resampling Methods, Bioinformatics, 2005, 21(15), pp. 3301-3307.