Gemstone Classification using Convolutional Neural Network

Gemstone Classification using Convolutional Neural Network

Thabitta Judixon

BEng (Hons) Software Engineering University of Westminster

London, UK

Raghu Shivaraman

Department of Computing Informatics Institute of Technology Colombo 06, Srilanka

Abstract – This study presents automatic image-based classification of 12 popular gemstones using Convolutional Neural Network. A comprehensive and diverse dataset consisting of 24,000 images was created by the author, with 70% for training, 20% for validation, and 10% for testing, where each gemstone had 1400 images for training, 400 images for validation, and 200 images for testing. Various preprocessing steps were applied, including resizing, renaming, removing duplicate images, data augmentation, and normalization. The CNN model was built by visualizing the training history of a model, specifically the loss and accuracy over epochs. The proposed system achieved an accuracy of 98% on the test set, with high precision and recall values for each class. The results of the study indicate that the proposed methodology is effective in accurately classifying gemstones, and can potentially be extended to other areas of study.

Keywords: Convolutional Neural Network, Gemstone, Data Augmentation

1. INTRODUCTION

   Gemstones are highly valued for their beauty, durability, and rarity. They are minerals, stones, or organic matter that can be cut and polished for use as jewelry or other ornamental objects. [1] The optical properties of gemstones, such as color, brightness, and transparency, make them highly sought after for various decorative purposes. Gemstones have been a fascination for humans since ancient times due to their rarity and high exchange value. [3]

   Gemstones have been an essential part of human adornment for centuries. The identification of different types of gemstones is crucial in the jewelry industry, where there is a need to classify and identify gemstones accurately and efficiently. Identifying a gemstone can be a complex task due to their nature and the similarity between several ores. Physical characteristics such as hardness, density, refraction of light, and color of the mineral contained in the stone are often used to identify the type of gemstone. According to experts, these four characteristics can be used to differentiate between different types of gemstones based on their physical properties.

   Currently, gemstone identification is done through visual observation and spectrochemical analysis, which is a difficult

   and time-consuming process. [2] With the emergence of new synthetic gemstones and treatment techniques and, human errors also occur during the identification process. Gemologists may misidentify a gemstone due to a lack of experience, insufficient knowledge, or the presence of similar-looking gemstones. This is particularly true for inexperienced gemologists who may struggle to distinguish between similar-looking gemstones. Moreover, fatigue, stress, and other factors can also affect the accuracy of identification, leading to serious consequences. Due to this reason, increasingly complex instruments have been introduced into gemological laboratories, but not all laboratories have access to these sophisticated instruments. Therefore, automatic techniques based solely on images are becoming increasingly attractive. With the advancement of computers and algorithms, image processing and computer vision tasks have become common place in many areas. In this context, the use of deep learning models for automatic gemstone classification based on images has gained attention. The goal is to develop an accurate and efficient system that can identify and classify gemstones, regardless of their origin, treatment, or shape.

   The advancement in computer vision and deep learning techniques, automatic gemstone identification has become a feasible approach. This study proposes a convolutional neural network-based approach for the automatic classification of 12 popular gemstones. The CNN model is trained on a comprehensive and diverse range of 24,000 images, ensuring high accuracy in gemstone identification. The model's training history, specifically the loss and accuracy over epochs, was visualized to optimize the model's performance. The proposed approach's accuracy was evaluated, and the results demonstrated in the CNN model achieved an accuracy of 98%, which is significantly higher than that of human experts. This research provides a promising approach to automating gemstone identification and can have practical implications in the jewelry industry. These results outperformed the classification accuracy of human experts and demonstrate the superiority of computer vision-based approaches for gemstone identification. The proposed methodology can be extended to other domains that require automatic image-based classification.

2. MATERIALS AND METHODS

   The selection of gemstones for this study was based on their popularity and widespread use in jewelry. The 12 gems chosen, namely Amethyst, Ametrine, Black Spinel, Blue Sapphire, Citrine, Diamond, Emerald, Pearl, Peridot, Rose Quartz, Ruby, and Smoky Quartz, are some of the most commonly used and well-known gems in the jewelry industry. [4] These gems are widely recognized for their unique physical and chemical properties, as well as their aesthetic value, and are therefore of significant interest to both gemologists and consumers alike. By choosing these 12 gems, this study aims to contribute to the development of a deep learning-based gemstone classification system that can accurately identify and classify popular and commonly used gems in jewelry.

3. MATERIALS

   The dataset used in this research includes images of 12 popular and commonly used gems in jewelry, including Amethyst, Ametrine, Black Spinel, Blue Sapphire, Citrine, Diamond, Emerald, Pearl, Peridot, Rose Quartz, Ruby, and Smoky Quartz. [4] The images were collected from various sources, including gemstone-related websites, jewelry shops, and other online marketplaces and carefully curated to ensure high-quality and consistent representation of each gem.

   Each gemstone has its own unique physical and optical properties [12], which are reflected in their appearance in the images. For example, Amethyst typically has a purple coloration, while Citrine has a yellow to orange hue. The images in the dataset capture these distinctive features, as well as variations in color, shape, and texture. Overall, the dataset provides a comprehensive and diverse representation of these popular gemstones, allowing for the development and testing of robust machine learning models for gemstone classification. The given below are the twelve representative images of the gemstones in this work.

   Amethyst Ametrine Black Spinel

   Blue Sapphire Citrine Diamond

   Emerald Pearl Peridot

   Rose Quartz Ruby Smoky Quartz

   Figure 1

4. METHODS

   These methods were used to create a dataset of 24,000 images of 12 popular and commonly used gems in jewelry. The dataset was then used to train a CNN model for gemstone classification.

   Data collection: The images of 12 gemstones – [19] Amethyst, Ametrine, Black Spinel, Blue Sapphire, Citrine, Diamond, Emerald, Pearl, Peridot, Rose Quartz, Ruby, Smoky Quartz were collected from various sources such as gemstone-related websites, jewelry shops, and gemstone collectors. [9] [17] [18]

   Data preprocessing: The images collected were resized to a uniform size of 256×256 pixels using OpenCV's resize function. The original images were of different sizes, and this step helped standardize the images for model training. Additionally, the pixel values of the resized images were normalized by dividing each pixel value by 255. [5] This step ensures that each pixel value ranges from 0 to 1, which is important for neural networks. The resizing and normalization were performed using Python's NumPy and OpenCV libraries. Additionally, duplicated images were removed from the dataset to avoid bias. Then, low quality, unwanted files and blurred images were discarded to ensure that the training data was of high quality. Only JPEG and PNG images were used, as these are the most common image formats used for uploading images on user interfaces. Finally, all the images were labeled according to the class they belonged to, and were renamed to follow a consistent naming convention. [7]

   Data augmentation: To increase the size of the dataset and reduce over fitting, data augmentation techniques were applied to the images. This involved randomly applying transformations such as rotations, zooming, and horizontal/vertical flipping to the images. [6]

   Dataset splitting: The dataset was split into training, validation, and testing sets. The training set consisted of 70% of the total images, while the validation test consisted 20% and testing sets consisted of 10%. [8]

5. ALGORITHM

   A Convolutional Neural Network model was built to classify the 12 types of gemstones in the dataset. The model was built using the Keras deep learning library and consists of several convolutional layers, followed by pooling layers, and finally, fully connected layers. [13] The CNN model was trained on the training set and its hyperparameters were tuned using visualizations of training history, specifically loss and accuracy over epochs. The output layer is a dense layer with 12 units, one for each class, and SoftMax activation function that returns the probabilities of the input image belonging to each of the 12 classes. The model uses L2 regularization on the weights of all the layers to prevent over fitting. It also uses the Adam optimizer with a learning rate of 0.0001 to minimize the categorical cross-entropy loss function. The model architecture has a total of 10 layers, including 5 convolutional layers, 3 fully connected layers, and 1 output

   layer. [15]

   Figure 2

   The accuracy of the model is also shown in the below, which is the proportion of correctly classified samples over the total number of samples. The accuracy for this model is 0.98, which indicates that the model is performing well overall.

6. ACCURACY

   The model accuracy is a measure of how well the model is able to correctly classify images. In this case, the model was trained on a dataset of 24,000 images belonging to 12 different classes and the model was tested on 2400 unseen images. This allows for an unbiased evaluation of the model's performance on new, unseen data. The accuracy of the model is reported to be 97.79%. [11]

   This defines that the model was able to correctly classify 97.79% of unseen images. This is a good indication that the model is performing well on this particular dataset without any over fitting.

   In addition to accuracy, other metrics such as precision, recall, and F1 score are also reported. Precision measures the proportion of true positive predictions out of all positive predictions. Recall measures the proportion of true positive predictions out of all actual positive instances. F1 score is the harmonic mean of precision and recall. In this case, the precision is reported to be 0.978490895428702, which means that 97.85% of the positive predictions made by the model were correct. The recall is reported to be 0.9779166666666668, which means that 97.79% of the actual positive instances were correctly identified by the model. The F1 score is reported to be 0.9779173581980395, which is the harmonic mean of precision and recall and provides an overall measure of the model's performance. Overall, the reported metrics indicate that the model is performing well on this particular dataset and is able to accurately classify images with a high degree of precision and recall. [20]

   Figure 3

   The first graph shows the training and validation loss of a deep learning model during the training process.

   Figure 4

   The second graph shows the training and validation accuracy of a deep learning model during the training process.

   Figure 5

7. EXISTING WORK

   The system utilizes DL, a research area that focuses on building systems that can gain knowledge, to classify gemstones. To improve the accuracy of the model, various techniques were explored, including DA, image clipping, and TL with the InceptionV3 model compared to other models found in previous research. Performance was evaluated based on accuracy and loss metrics, and when available, F1, Precision, and Recall metrics were also used. A dataset of 87 gemstone categories was used for the experiments, and the proposed model achieved a 72% accuracy, indicating the potential for automating gemstone classification using DL techniques. [10]

   The aim of this study is to develop a system that can recognize three types of gemstones, namely Ruby, Sapphire, and Emerald, by using the HSV color space, image processing techniques, and an ANN backpropagation algorithm. The system will be trained using the hue values from the HSV color space of the gemstones. In the image processing step, the system will crop, resize, convert from RGB to HSV, extract the hue color, and represent it as a 30×3 matrix. The ANN model will consist of three input layers, three hidden layers, and one output layer with pre-determined targets. The system achieved an accuracy of 90.66% with five rounds of training and 25 rounds of testing for any type of gemstone. This result suggests that the use of an Artificial Neural Network Backpropagation algorithm is a successful approach for identifying gemstone types because of the high level of accuracy. [3]

   This study introduces a method for automatically classifying gemstones based on their images using computer vision. The dataset contained 2042 training images and 284 test images that were divided into 68 categories of gemstones. The authors used 33 different feature extraction techniques, including color histograms in different color spaces, local binary pattern, Haralick texture, and grey-level co-occurrence matrix properties, combined with various machine learning algorithms such as Logistic Regression, Linear Discriminant Analysis, K-Nearest Neighbour, Decision Tree, Random Forest, Naive Bayes and Support Vector Machine. They also tried using deep learning classification with ResNet-18 and ResNet-50. The best results were achieved with the Random Forest algorithm, which used an eight-bin RGB color histogram and local binary pattern features, achieving an accuracy of 69.4% on the test images in just 0.0165 seconds. These results were compared to three expert gemmologists with over five years of experience in gemstone identification, who obtained accuracies between 42.6% and 66.9% and took between 42 and 175 minutes to classify the test images. [16]

   1. EVALUATION

      The proposed system achieves a 98% accuracy using CNN and a dataset of 12 gemstone classes that was created specifically for this research. This outperforms other systems

      and demonstrates the potential for automated gemstone classification using deep learning techniques. Because the development of the dataset fills a gap in the research, as there was no proper dataset available for this purpose, and other existing system used Kaggle dataset which is insufficient to train a deep leaning model. This adds significant value to the research, highlighting the potential for further advancements in this field. The proposed research achieved a significantly higher accuracy compared to the existing systems due to thorough data preprocessing and a large dataset of 24,000 clear images, providing sufficient data for deep learning algorithms. These factors likely contributed to the superior performance of the CNN-based gemstone classification system, demonstrating the potential of advanced techniques for automated gemstone identification.

   2. CONCLUSION

      In conclusion, this research proposes a CNN-based system for automated gemstone classification, which achieved a accuracy of 98% in classifying 12 different gemstone classes. The use of a custom dataset, data preprocessing techniques, and clear images in training, combined with the CNN model, were the key factors that contributed to the high accuracy achieved. The results demonstrate the potential for further advancements in automated gemstone classification, and the importance of creating high-quality datasets for deep learning applications. [14] Overall, this research highlights the significance of deep learning techniques in gemstone classification and suggests avenues for future research in this field.

   3. FUTURE ENHANCEMENTS

      A possible approach could be the use of a hybrid algorithm that combines deep learning techniques with traditional computer vision algorithms. This could potentially improve the accuracy of the system by leveraging the strengths of both approaches. Additionally, expanding the dataset to include a larger number of gemstone types, especially those with identical appearances, could help to further improve the system in real-world scenarios.

      Acronym	Description
      DL	Deep Learning
      CNN	Convolutional Neural Network
      DA	Data Augmentation
      TL	Transfer Learning
      ANN	Artificial Neural Network
      HSV	Hue Saturation Value

   4. ABBREVIATIONS

REFERENCES

[1] [1] Mol, A.A. et al. (2007) Efficiency parameters estimation in gemstones cut design using artificial neural networks, Computational Materials Science, 38(4), pp. 727736. Available at: https://doi.org/10.1016/j.commatsci.2006.05.012.

[2] [2] Shigley, J.E. (2008) A review of current challenges for the identification of gemstones, Geologija, 50(4), pp. 227236. Available at: https://doi.org/10.2478/v10056-008-0048-8.

[3] [3] Maula, I. et al. (2018) Development of a Gemstone Type Identification System Based on HSV Space Colour Using an Artificial Neural Network Back Propagation Algorithm, 149(Icosat 2017), pp. 104109. Available at: https://doi.org/10.2991/icosat-17.2018.24.

[4] [4] Policy, K.M.-W.K.M.-W.K. is an experienced writer who has written hundreds of articles for publication R.M.L. about our E. (n.d.). List of Gemstones: Guide to 18 Common Gems. [online] LoveToKnow. Available at: https://www.lovetoknow.com/life/style/list-gemstones [Accessed 19 May 2023].

[5] [5] Geisler Mesevage, T. (2021). What Is Data Preprocessing & What Are The Steps Involved? [online] MonkeyLearn Blog. Available at: https://monkeylearn.com/blog/data-preprocessing/.

[6] [6] TensorFlow. (n.d.). Data augmentation | TensorFlow Core. [online] Available at: https://www.tensorflow.org/tutorials/images/data_augmentation.

[7] [7] www.javatpoint.com. (n.d.). Data Preprocessing in Machine learning – Javatpoint. [online] Available at: https://www.javatpoint.com/data-preprocessing-machine- learning.

[8] [8] Filter, J. (n.d.). split-folders: Split folders with files (e.g. images) into training, validation and test (dataset) folders. [online] PyPI. Available at: https://pypi.org/project/split-folders/.

[9] [9] gemselect.com. (n.d.). Gemstones by Color: A Color Guide for Gemstones with Images and Details. [online] Available at: https://www.gemselect.com/gem-info/gems-by-color.php [Accessed 19 May 2023].

[10] [10] Freire, W.M., Amaral, A.M.M.M. and Costa, Y.M.G. (2022) Gemstone classification using ConvNet with transfer learning and fine-tuning, International Conference on Systems, Signals, and Image Processing, 2022-June, pp. 1821. Available at: https://doi.org/10.1109/IWSSIP55020.2022.9854441.

[11] [11] freeCodeCamp.org. (2021). How to Improve the Accuracy of Your Image Recognition Models. [online] Available at: https://www.freecodecamp.org/news/improve-image- recognition-model-accuracy-with-these-hacks/.

[12] [12] Ganoksin. (n.d.). Optical and Physical Properties of Gemstones – Ganoksin Jewelry Making Community. [online] Available at: https://www.ganoksin.com/article/optical-and- physical-properties-of-gemstones/.

[13] [13] upGrad blog. (2022). Basic CNN Architecture: Explaining 5 Layers of Convolutional Neural Network. [online] Available at: https://www.upgrad.com/blog/basic-cnn- architecture/#:~:text=other%20advanced%20tasks.-.

[14] [14] Mathworks (2019). What Is Deep Learning? | How It Works, Techniques & Applications. [online] Mathworks.com. Available at: https://www.mathworks.com/discovery/deep- learning.html.

[15] [15] Mandal, M. (2021). CNN for Deep Learning | Convolutional Neural Networks (CNN). [online] Analytics Vidhya. Available at: https://www.analyticsvidhya.com/blog/2021/05/convolutional- neural-networks-cnn/.

[16] [16] Chow, B.H.Y. and Reyes-Aldasoro, C.C. (2022) Automatic gemstone classification using computer vision, Minerals, 12(1). Available at: https://doi.org/10.3390/min12010060.

[17] [17] www.rasavgems.com. (n.d.). Wholesale gemstone manufacturer & supplier online- RASAVGEMS. [online] Available at: https://www.rasavgems.com/ [Accessed 19 May 2023].

[18] [18] Minerals.net. (2019). The Mineral and Gemstone Kingdom: Home. [online] Available at: https://www.minerals.net/.

[19] [19] Gem Rock Auctions. (2019). Gem Auctions Online – Verified Gemstone Dealers – Gemstones For Sale | Gem Rock Auctions. [online] Available at: https://www.gemrockauctions.com/.

[20] [20] www.labelf.ai. (n.d.). What is Accuracy, Precision, Recall and F1 Score? [online] Available at: https://www.labelf.ai/blog/what-is-accuracy-precision-recall- and-f1-score [Accessed 19 May 2023].