Recognition of food type and calorie estimation

Recognition of food type and calorie estimation

ISSN: 2278-0181

Muhammed Suhair K Assistant Professor, CSE MEA Engineering College, Perinthalmanna, India suhair@meaec.edu.in

Harshad ME Computer science and engineering

MEA Engineering College Perinthalmanna, India 19mcs54@meaec.edu.in

Harsha PC

Computer science and engineering MEA Engineering College Perinthalmanna,India 19gcs13@meaec.edu.in

Danish Babin Computer science and engineering

MEA Engineering College Perinthalmanna,India 19mcs30@meaec.edu.in

Habeeba M

Computer science and engineering MEA Engineering College Perinthalmanna, India 19gcs19@meaec.edu.in

AbstractPeople are becoming more mindful of their health and wish to live regular, healthy lives all around the world.But because of how quickly things are changing, obesity and other related problems have emerged as the biggest health concerns for people.A person is considered obese, in the opinion of medical professionals, when their BMI is higher than 30 kg/m2.Numerous illnesses, including high cholesterol, liver failure, difficulty breathing, heart difficulties, diabetes, and occasionally cancer, are brought on by obesity.We can reduce obesity in the population by consuming nutrient-dense, low- calorie foods .Many people struggle with making food selections. So it aims to develop a system for recognizing different types of food and estimating their calorie content and nutrients using deep learning techniques . The proposed system utilizes a convolutional neural network (CNN) architecture to extract features from food images and their corresponding calorie values. It also includes a patient menu separately designed for different patients like those with chronic kidney disease (CKD), diabetes melitus, heart disease, etc. The system achieves high accuracy in recognizing food types and estimating their calorie and nutrient content.

KeywordsDeep learning, convolutional neural network, Calorie estimation , Patient Menu

1. INTRODUCTION

   A lot of individuals enjoy consuming junk food and soft drinks, which are high in calories and sugar.There is an increase in obesity due to people's unregulated eating habits, lack of exercise, ignorance of dietary foods, and lack of awareness about them.Obesity-related conditions include hypertension, diabetes, heart problems, breathing difficulties, and others. Obesity is mostly caused by unbalanced dietary food habits and eating foods that are higher in calories and lower in nutrients. Having a balanced diet, engaging in regular physical activity, and maintaining good eating habits can lead to a healthy life with a normal BMI.

   In this system, recognition of food type and calorie estimation of people and patients using deep learning is a cutting-edge technology that utilizes artificial intelligence to accurately provide the type of food that is to be consumed

   and estimate the calorie in the meal. The project involves training deep learning models using food 101 datasets which includes different varieties of 101 food and patient menu dataset which includes varieties of food that a patient can intake. The models use convolutional neural network algorithms to recognize the food in the images and estimate its calorie content based on various factors such as portion size, ingredients, etc.

   The primary objective of this system is to provide a more accurate and efficient way of tracking patients dietary intake and normal people dietary intake. This is particularly important for patients with certain health conditions such as diabetes, obesity, chronic kidney disease, heart disease,etc who need to closely monitor their calorie intake to manage their condition.

   Overall, the recognition of food type and calories estimation using deep learning system has the potential to revolutionize the way healthcare professionals monitor and manage the patient's dietary intake, youths fitness, ultimately improving patient outcomes and quality of life.

2. REVIEW OF RELEVANT LITERATURE

   1. Food Image Recognition and Calorie Prediction [1].

      This system receives photos of food items as input and outputs the food item's calories.In order to accomplish this, a number of intermediary processes are used.First, the detected food items in the collected image whose calories need to be forecasted are noted.Before they are employed in the model, food calories are calculated to 128*128 pixels once the food item's size, volume, and last step have been identified.For picture recognition, the Mask R-CNN algorithm is employed, and the approximate proportion method format is used to predict calories.

   2. Calorimeter: Food Calorie Estimation using Machine Learning [2].

      In this model, food size is analyzed through various machine learning modules. The user selects two images of the dish, one from the front and one from the top, by clicking on a

      picture of the food. The machine uses these photos as input and employs various algorithms to determine the volume and serving size. This can be accomplished through multiple machine learning techniques, with the option to define and

      resource for mobile nutrition applications as ItShSeNd: a2t2a78w-0a1s81

      collected in real-life settiInCgCs IuDsiTng- 2s0m2a3rCtpohnofnereesn. ce Proceedings

      segment the image simultaneously. The end result provides 6 6. Image-Based Food Calorie Estimation Using Knowledge

      a comprehensive summary of food items, including calorie and nutrient information, and the entire process is automated.

   3. Fine-Grained Food Classification Methods on the UEC FOOD-100 Database [3].

      This work strives to advance automatic food recognition by presenting the most widely used food classification algorithms, highlighting the most significant food item databases available, and achieving state-of-the-art performance in the top classification experiment of the UEC Food-100 database. Specifically, the article raises the current best-shot performance to 90.02% by a 0.44

      of Food Categories, Ingredients, and Cooking Directions [6].

      The article focuses on a strategy for directly measuring food calories by using a multi-task CNN to simultaneously estimate food categories, ingredients, and cooking instructions. This approach is believed to improve performance compared to training them separately. The calorie estimation is treated as a regression problem, taking in a food image and producing an output of the food's calorie count per serving, based on the assumption that a food photo represents only one dish. The authors follow the work of other researchers, who suggested using a multi-task CNN to simultaneously estimate food categories and ingredients.

      percentage point increase. As far as we know, this is also the 7 7. Application of Deep Learning in Food: A Review [7].

      first publication to compare the average results of a

      classification experiment performed on the UEC Food-100 database over five trials. The average performance, as expected, is slightly lower than the best-shot performance.

   4. Refined image segmentation for calorie estimation of multiple dish food items [4].

   This article introduces a method that utilizes computer vision and deep learning to predict the calorie content of dishes using top-view images of food. The system conducts a fine- tuned image segmentation process through the use of convolutional neural networks (CNNs), which mimic instance segmentation by assigning each pixel to the instance of objects in an image. Research in semantic segmentation and object detection is being caried out with the use of CNNs. The identified pixels create masks or segmentations that correspond to different food categories. The volume and mass of the detected food items are then estimated using a reference object and these output masks. The calorie content of the food items is determined by combining the estimated volume, mass, and other relevant data.

   5. Multi-Task Learning for Calorie Prediction on a Novel Large-Scale Recipe Dataset Enriched with Nutritional Information [5].

   In this work, calorie estimation is the main focus, but the dataset also includes other meal characteristics such as type of food, ingredient amounts, cooking directions, user rating, preparation time, tags, and additional attributes. This information can be used to improve the calorie prediction models in the future, such as enhancing the phrase embedding model with the cooking instructions as a text corpus. The pic2kcal dataset can also be used to address other food computing issues, such as determining the dietary properties of a meal, and is expected to be a valuable

   In this study, the author conducted a review of recent studies on the application of deep learning in food data interpretation, including photos, spectra, words, and other information. The study covers the design, training methods, and evaluation results of the deep neural networks used in each of the surveyed papers. The results show that deep learning outperforms other commonly used methods in the field. The author also discusses the challenges and potential uses of deep learning in the food industry and provides an analysis of the advantages and disadvantages of these techniques. This study is reportedly the first review of deep learning applications in the food industry.

   8. Food calorie measurement using deep learning neural network [8].

   In this research, I suggested employing CNNs to simultaneously learn food calories, categories, ingredients, and cooking instructions in order to estimate food calories from a meal photo. Additionally, we created two distinct datasets: one of calorie-annotated dishes gathered from Japanese recipe websites on the Internet, and the other of calorie-annotated recipes gathered from an American recipe website. In these tests, multi-task CNNs exceeded the results of separate single-task CNNs in both datasets.

   9. Food Calorie Estimation using Convolutional Neural Network [9].

   The suggested model uses deep learning techniques to offer a special method of calculating calories. In the world of medicine, calorie calculations for food are crucial. because the calories in this dish are healthy. This measurement is derived from photos of food seen on various fruits and vegetables. A neural network is used to take this measurement. One of the greatest ways to categorize machine learning techniques is with the tensor flow. Through the use of a convolutional neural network, this method calculates the calories in meals. An image of food is used as the input for this calculated model. With food object

   identification, the suggested CNN model calculates the food's caloric value.Volume error estimation serves as the primary parameter for the outcome, while calorie error estimation serves as the second parameter. A 20% reduction in volume estimation error is made gradually. This shows that the proposed CNN model offers a higher accuracy level than the current model.

   1. Recognition of food type and calorie estimation using a neural network [10].

      The image of the fruits, vegetables, or other food is used as the system's input in this system. The system has two phases: testing comes after training. The classifier is trained to produce the datasets during the training phase, which involves extracting the necessary characteristics from the input photos. To acquire the region of interest for which the nutritional value needs to be determined, the input image is scaled and segmented. Using the multilayer perceptron (MLP), features including size, shape, and texture are retrieved from the segmented image and stored as datasets for use in the actual testing phase.

3. PROPOSED WORK

   The Food 101 dataset and the Patient Menu dataset are used as input. In this proposed system, we use the convolutional neural network algorithm for estimating the calories and nutrients present in the food. A convolutional neural network is a type of deep learning algorithm commonly used for image recognition tasks such as food type and calorie estimation.

   The basic architecture of a convolutional neural network consists of multiple layers, including

   • Input layer – Responsible for receiving raw image data

   • Convolutional layers, -Actual image recognition takes place, consist of a set of filters that are applied to the input images.

   • Pooling layers – Downsample the output of the convolutional layer by summarizing the features extracted by filters.

   • Fully connected layers – Responsible for performing classification.

   • Output layer – Produces predicted values

   ISSN: 2278-0181

   ICCIDT – 2023 Conference Proceedings

   Figure 3.1: Proposed system

   • Steps to recognize and estimate calories and nutrients

     • Data Collection:The first step in building a deep learning model for food recognition and calorie estimation is to gather a large data set of food images, along with corresponding calorie information. In this, we use the food 101 dataset and the patient menu dataset.

     • Data preprocessing: This will involve tasks such as resizing and cropping images, normalizing color channels, and computing data into a format.

     • Convolutional neural network: Using this algorithm, it will extract features from input images.

     • Training and optimization: The input dataset images are trained for further steps.

     • Recognition of image: The trained image is recognized.

     • Calorie and nutrient estimation: The calorie and nutrient content of food is estimated.

   Figure 3.2: Data flow diagram of proposed system – level 1

   ISSN: 2278-0181

   ICCIDT – 2023 Conference Proceedings

   Figure 3.3: Data flow diagram of proposed system – level 2

   . A.Overview

4. METHODOLOGY

   Our model accepts the food item's image as input and outputs the food item's calories. In order to accomplish this, a number of intermediary processes are used. First, the detected food items in the collected image whose calories need to be forecasted are noted. For picture recognition, the CNN algorithm is employed, and the approximate proportion method format is used to predict calories.

   1. Dataset

      There are two different datasets used. They are,

      • Food 101 dataset

        • It includes 101 types of food.

      • Patient menu dataset

        • It includes foods that can be eaten by different types of patients, like those with chronic kidney disease, diabetes melitus, heart patients, etc.

          By choosing photos of food items from the patient menu dataset and from datasets like Food 101, some of the foods are apple pie, club sandwich, waffles, rice, chicken curry, and idli.

          Food Item	Image Count
          Apple pie	113
          Club sandwich	123
          Waffles	101
          Rice	100
          Chicken curry	126
          Idli	105

   2. Food-Item Identification

      We can browse the food image from the data set and estimate the calories and nutrients of the chosen food item. and capture the live image of the food item and identify its calories.

   3. Food Calorie Estimation

      The calorie and nutrient values of chosen or captured images are predicted.

      Figure 4.1: Waffles

      Nutrition Fact (per 100 g)

      	Estimated
      Calories	299.3
      Total Fat	14.0
      Saturated fat	2.9
      Carbohydrates	32.3
      Protein	7.8
      Sodium	501
      Potassium	192
      Cholesterol	68

5. RESULT

   Accurate food recognition: With the use of deep learning algorithms, an application can recognize different types of food with a high degree of accuracy. This can be helpful for individuals who are trying to monitor their calorie intake and ensure they are meeting their nutritional goals.

   Calorie estimation: Deep learning models can also estimate the number of calories in a particular food item based on its appearance and nutritional content. This can help individuals track their calorie intake more accurately and make informed decisions about their diet.

   ISSN: 2278-0181

   ICCIDT – 2023 Conference Proceedings

   Patient menu calorie estimation: Patients with chronic kidney disease, diabetes melitus, heart disease, obesity, etc. can login to the application and estimate the calorie of the food item that can be consumed by them.

6. CONCLUSION

In conclusion, recognition of food type and calorie estimation using deep learning applications can have a significant impact on the field of nutrition and health. It can provide accurate food recognition, calorie estimation, patient menu calorie estimation, improved health outcomes, and valuable data collection. We may create several patient menus, allowing patients to log in and see what they can eat and how many calories and nutrients it contains.

Although there are some obstacles that require attention, including the absence of consistent food databases and variations in serving sizes and cooking techniques, the precision of the models can be impacted by the quality and diversity of the input data, as well as the intricacy of the food that is being examined. Nevertheless, the potential for deep learning models to enhance the precision and productivity of food recognition and calorie estimation is significant, and can have valuable implications in the

domains of nutrition and healthcare. Continuing research

(ICCCIS). IEEE, 2021.

ISSN: 2278-0181

and advancements in this field can lead to more sophisticated and efficient models that can be practically applied.

REFERENCES

[1] Narayana, et al, and Darapaneni "Food Image Recognition and Calorie Prediction." 2021 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS). IEEE, 2021.

[2] Pramod B, Deshmukh, Rahul Y. Pawar, and Vishakha

A. Metre, and. "Calorimeter: Food Calorie Estimation using Machine Learning." 2021 International Conference on Emerging Smart Computing and Informatics (ESCI). IEEE, 2021.

[3] Arslan, Berker, et al. "Fine-grained food classification methods on the UEC food-100 database." IEEE Transactions on Artificial Intelligence 3.2 (2021): 238-

243.

[4] J. Angel Arul Jothi, Poply, and Parth. "Refined image segmentation for calorie estimation of multiple-dish food items." 2021 International Conference on Computing, Communication, and Intelligent Systems

[5] Robin, et al, and RueIdCeC"IMDTul-ti2-0ta2s3kCleoanrfnerinegncfeoPr rcoacloeerdieings prediction on a novel large-scale recipe dataset enriched with nutritional information." 2020 25th International Conference on Pattern Recognition (ICPR). IEEE,

2021.

[6] Keiji Yana and Takumi Ege, Image-Based Food Calorie Estimation Using Knowledge on Food Categories, Ingredients, and Cooking Directions 2021 The University of Electro-Communications, Tokyo.

[7] Fei Liu, Chu Zhang, Lei Zhou, Yong He, and Zhengjun Qiu Application of Deep Learning in Food: A Review 2021, Comprehensive Reviews in Food Science and Food Safety.

[8] Shervin, Pallavi, Sri Vijay, Abdulsalam Yassine, Parisa. Food calorie measurement using deep learning neural network 2021, research gate

[9] V. Balaji, N. Jayapandian, and Kasyap Food Calorie Estimation using Convolutional Neural Network." 2021 3rd International Conference on Signal Processing and Communication (ICPSC). IEEE, 2021.

[10] R. Dinesh, et al, and Kumar "Recognition of food type and calorie estimation using neural network." The Journal of Supercomputing 77.8 (2021): 8172-8193.