Implementation of OpenCV Model for Inward Inspection Technique

Implementation of OpenCV Model for Inward Inspection Technique

Abhishek Satkar Department of Mechanical Engineering Vishwakarma Institute of Technology

Pune, India

Siddhesh Jejurkar Department of Mechanical Engineering Vishwakarma Institute of Technology

Pune, India

Yashkumar Shinde Department of Mechanical Engineering Vishwakarma Institute of Technology

Pune, India

Prof. (Dr.) Laxmikant Mangate Department of Mechanical Engineering Vishwakarma Institute of Technology Pune, India

Abstract–The main aim of this project is to automate inspection process with an efficient yet cost effective equipment which would have a target market as the small-scale industries. Many industries prefer manual inspection by workers due to the high cost of machines that are already in the market and we want to provide an alternative. There are many benefits to this project some of which are high efficiency, low cost, simple design and compact design. Automating the system will provide a run time of 24 / 7 with the same accuracy. This project will allow all industries a chance to include automation and cut costs. The project includes object detection using Computer Vision. We are aiming at minimum 95% accuracy in object detection. Parameters of the component will be measured using Computer Vision and would be accepted or rejected using Computer Vision.

Keywords–Inspection, OpenCV, Arduino, Conveyor, Linear Actuator

1. INTRODUCTION

   The requirement of product quality inspection in industries for product standardized leads to a development of the quality inspection system. The project presents an automated real- time vision quality inspection monitoring system as a problem solver to a manual inspection that is tedious and time-consuming task as well as reducing cost especially in small and medium enterprise industries (SME).

   The analysis includes pre- processing, color concentration using the histogram and quadratic distance and depth inspection using coordinate vertical and horizontal reference levels. The project focusses on implementing Computer Vision for accepting or rejecting parts that come from the supplier if it matches the required dimensions. This system is cost effective and will help in achieving maximum accuracy and efficiency. Our main goal is to build a system which is cost effective and industries of all scale and shape can implement it with utmost ease.

   There are two parts to our project

   1. Computer Vision

   2. Mechatronics

   Both need to work hand in hand to have a functional and implementable system.

2. LITURATURE REVIEW

   1. A Bevel Gear Quality Inspection System Based on Multi- Camera Vision Technology

      Surface defect detection and dimension measuring of automotive bevel gears by manual examination area unit pricey, inefficient, low speed and low accuracy. so as to resolve these issues, an artificial gear quality examination system supported multicamera vision technology is developed. The system will find surface defects and live gear dimensions at the same time. 3 economical algorithms named Neighborhood Average distinction (NAD), Circle Approximation technique (CAM) and quick Rotation- Position (FRP) area unit projected. The system will find knock harm, cracks, scratches, dents, excrescence or recurrent cutting of the spline, etc. the littlest detectable defect is zero.4 mm × 0.4 millimeter and therefore the exactitude of dimension measuring is regarding four050 m. One examination method takes no quite one.3 s. each exactitude and speed meet the necessities of period on-line examination in gear production.

      Fig. 1. Diagram of bevel gear inspection system

   2. R-CNN-Based Large-Scale Object-Defect Inspection System for Laser Cutting in the Automotive Industry

      A car side-outer is an iron mold that is applied in the design and safety of the side of a vehicle, and is subjected to a complicated and detailed molding process. The side outer has

      three features that make its quality inspection difficult to automate: (1) it is large; (2) there are many objects to inspect; and (3) it must fulfil high-quality requirements. Given these characteristics, the industrial vision system for the side outer is nearly impossible to apply, and indeed there is no reference for an automated defect-inspection system for the side-outer. Manual inspection of the side-outer worsens the quality and cost competitiveness of the metal-cutting companies. To address these problems, we propose a large-scale Object- Defect Inspection System based on Regional Convolutional Neural Network (R-CNN; RODIS) using Computer Vision (AI) technology. In this paper, we introduce the framework, including the hardware composition and the inspection method of RODIS. We mainly focus on creating the proper dataset on-site, which should be prepared for data analysis and model development. Additionally, we share the trial-and- error experiences gained from the actual installation of RODIS on-site. We explored and compared various R CNN backbone networks for object detection using actual data provided by a laser cutting company. The Mask R-CNN models using Res-net-50-FPN show Average Precision (AP) of 71.63 (Object Detection) and 86.21 (Object Seg- mentation), which indicates a better performance than that of other models.

   3. Automated Real-Time Vision Quality Inspection Monitoring System

   The requirement of product quality inspection in industries for product standardized leads to a development of the quality inspection system. The problem is related to a manual inspection that is done by a human as an inspector. This paper presents an automated real-time vision quality inspection monitoring system as a problem solver to a manual inspection that is tedious and time-consuming task as well as reducing cost especially in small and medium enterprise industries (SME). For the proposed system, soft drink is used as the test product for quality inspection. The system uses computer- network to inspect two quality inspections which are color concentration and water level. The analysis includes pre- processing, color concentration using the histogram and quadratic distance and level inspection using coordinate vertical and horizontal reference levels. This paper presents an automated real-time vision quality inspection monitoring system. The inspection is broadly determined into two categories which are color and level classification. Color classification introduces a quadratic distance technique to classify color of the beverage product based on the histogram analysis. Besides, raspberry pi is used to control IR sensor and DC motor at the system. The captured image is stored in the Apache server so that the image can be replaced at any time. The performance is verified based on the system accuracy to inspect colors and levels of the soft drink bottle. The results show this system can be used to classify color and level of the beverage product. Thus, this system has the ability to solve problems of quality control, especially for Small and Medium (SME) industries.

3. METHODOLOGY

   Fig.2. Connection Schematic

   Motor is controlled using L298N motor control driver which supports up to 12V of Dc supply and can support up to 4 36V motors. L298N is able to take input from digital pins on Arduino Uno. Here, N3 and N4 pins on l298N are connected to digital pins 6 and 7 on Arduino Uno respectively. External power supply of 12V is provided to L298N motor driver by connecting to positive and negative terminals to +12V and GND terminals respectively. 12v DC 100rpm moto is connected to L298N in Output 3 and 4 terminals respectively.

   Fig. 3. Setup

   1. Camera Sensor sends live feed to the laptop wherein an OpenCV code determines the size of the object.

   2. If the object size is not within the tolerance limit, serial Library from OpenCV sends a binary signal to the Arduino Uno.

   3. Arduino Uno then transmits the binary signal through digital pins to L298N.

   4. L298N, a motor driver, consisting of an H-bridge, helps in reversing the polarity to actuate the actuator back and forth.

   5. L298N and 12V DC motor (which drives the conveyor belt) are connected in parallel to 12V 2A Power Adaptor.

      1. Organization of Dissertation

         The projects main focus is to convert a manual operation into automation. Some small-scale industries often manually measure components for the dimensional accuracy when sourced from a third-party vendor before using it for further processes. Based on our experience in one such industry, we have decided to automate the process of measuring gear parameters like inner diameter and outer diameter using Computer Vision. Also, such automated systems do exist but

         are very costly. We aim to build a system that is cost effective and usable for small scale industries.

         Setup Idea Conveyor Belt with a screening area, then a accept/reject system using pneumatic or linear actuators

      2. Project Design

         There are two parts to our project

         1. Computer Vision

         2. Mechatronics

         Various components related to each are mentioned in the flowchart.

         Fig. 4. Components

      3. Components used for Project

         1. Arduino UNO

            Arduino UNO is a microcontroller board based on the ATmega328P. It has

            14 digital input/output pins (of which 6 can be used as PWM outputs), 6 analog inputs, a 16 MHz ceramic resonator, a USB connection, a power jack, an

            ICSP header and a reset button. Fig. 5. Arduino Arduino is an open-source physical computing platform based on a simple i/o board and a development environment that implements the Processing/Wiring language. Arduino can be used to develop stand-alone interactive objects or can be connected to software on your computer (e.g., Flash, Processing, MaxMSP).

         2. Motor Driver Module

            The L298N is a dual H- Bridge motor driver which allows speed and direction control of two DC motors at the same time. The module can drive DC motors that have voltages between 5 and 35V, with a peak current up to 2A. The module has two screw terminal blocks for the

            motor Fig.6.L298N

            A and B, and another screw terminal block for the Ground pin, the VCC for motor and a 5V pin which can either be an input or output. This depends on the voltage used at the motors VCC. The module has an onboard 5V regulator which is either enabled or disabled using a jumper. If the motor supply voltage is up to 12V we can enable the 5V regulator and the 5V pin can be used as output, for example for powering our Arduino board. But if the motor voltage is greater than 12V we must disconnect the jumper because those voltages will cause damage to the onboard 5V regulator. In this case the 5V pin will be used as input as we need connect it to a 5V power supply in order the IC to work properly.

         3. Motor for Linear Actuator

            DC Motor 100RPM 12Volts geared motors are generally a simple DC motor with a gearbox attached to it. This can be used in all-terrain robots and variety of robotic applications. These motors have a 3 mm threaded drill hole in the middle of the shaft thus making it simple to connect it to the wheels or any other mechanical assembly.

         4. Motor to Drive Conveyor

            It is a simple DC motor featuring metal gearbox for driving the shaft of the motor, so it is a mechanically commutated electric motor which is powered from DC supply. The Johnson Geared Motors are known for their compact size and massive torque-speed characteristic. The Johnson Motor comes with side shaft also known as an off-centered shaft and six M3 mounting holes. The shaft of the motor equips metal bushes which makes these DC gear motors Shaft wear resistant. The shaft of the motor has a hole for better coupling.

         5. Conveyor System

      4. Simulation

         Factory IO:

         Factory I/O is a 3D factory simulation for learning automation technologies. Designed to be easy to use, it allows to quickly build a virtual factory using a selection of common industrial parts. Factory I/O also includes many scenes inspired by typical industrial applications, ranging from beginner to advanced difficulty levels. The most common scenario is to use Factory I/O as a PLC training platform since PLC are the most common controllers found in industrial applications. However, it can also be used with microcontrollers, SoftPLC, Modbus, among many other technologies.

         Fig. 7a. Factory IO Simulation

         Fig. 7b. Factory IO Simulation

         In the above image, simulation of the project idea is done by using Factory IO software. The conveyor belt system is used with a vision sensor and a linear actuator. The Computer Vision Sensor gives the input to the actuator and decides whether to accept or reject the part.

         Code: This code demonstrates how to use CV to detect and measure the approximate size of assembly line parts. It is designed to work with an assembly line camera mounted above the assembly line belt. The application monitors mechanical parts as they are moving down the assembly line and raises an alert if it detects a part on the belt outside a specified size range. The size of the sample element is measured through a video source, such as a camera. The application captures video frames and processes the frame data with OpenCV algorithms. It detects objects on the assembly line and calculates the length and width of the objects. If the calculated length and width is not within a predefined range, the application raises an alert to notify the assembly line operator. Optionally, the application sends data to a message queuing telemetry transport (MQTT) machine, or machine messaging server, as a part of an assembly line data analytics system.

         Serial Communication between Python and Arduino Used for communication between the Arduino board and a computer or other devices. All Arduino boards have at least one serial port (also known as a UART or USART), and some have several.

4. EXPERIMENTAL VALIDATION

   Fig. 7c. Control IO

   Control IO is a module available in Factory IO which helps in building the control map of the simulation without the need of a PLC. In the above image, we have mentioned the logic used for the simulation. A NOT Gate has been used which takes input from Vision Sensor and Actuates the actuator accordingly. On the right, the power module is connected to all the conveyor belts. Factory IO also has modules which help in configuring PLCs and we plan to use that in our real project model.

   1. Code Development

   Open CV: OpenCV (Open-Source Computer Vision Library) is a library of programming functions mainly aimed at real- time computer vision. Originally developed by Intel, it was later supported by Willow Garage then Itseez (which was later acquired by Intel). The library is cross-platform and free for use under the open-source Apache 2 License. Starting with 2011, OpenCV features GPU acceleration for real-time operations. The main contributors to the project included a number of optimization experts in Intel Russia, as well as Intel's Performance Library Team.

   Fig. 8. Code Run -1

   In the above image, the code measures the dimensions of the nail. The dimensions are mentioned on the left along with area. The length of the above sample is in the acceptable range and hence the green colored box is displayed on the nail.

   Fig. 9. Code Run – 2

   Here, the dimensions of the nail are not in the acceptable range and hence a red box is displayed on thenail.

5. CONCLUSION

1. Code Development along with testing was successfully done using python with an accuracy of 97%.

2. The Open CV algorithm is successfully able to differentiate the object size within the tolerance of 1.5mm – 2mm.

3. The accuracy will be further improved by implementing multiple camera setup and high- resolution cameras.

4. A Rack and Pinion mechanism driven by a 12V DC motor is used as an actuator to separate the rejected parts from the lot.

5. The further speed of actuation can be improved by using a pneumatic arm as an actuator instead of rack and pinion mechanism.

ACKNOWLEDGMENT

We express our deepest gratitude and heartfelt thanks to our guide, Prof. Dr. L. D. Mangate for his expert guidance, constant encouragement, constructive criticism, and inspiring advice throughout the completion of this report. We are thankful to the Mechanical Engineering Department, VIT Pune who provided us this opportunity to explore the topic through this evaluation component. Last but not least we would like to thank Prof. Dr. R M Jalnekar sir for giving us these opportunities.

REFERENCES

[1] Shao, Y.M.; Liang, J.; Gu, F.S.; Chen, Z.G.; Ball, A. Fault prognosis and diagnosis of an automotive rear axle gear using an RBF-BP neural network. In 9th International Conference on Damage Assessment of Structures; Ouyang, H., Ed.; IOP Publishing: Bristol, UK, 2011; Volume 305, pp. 613623. [Google Scholar] [2] Parey, A.; Tandon, N. Impact velocity modelling and signal processing of spur gear vibration for the estimation of defect size. Mech. Syst. Signal Process. 2007, 21, 234243. [Google Scholar] [CrossRef] [3] Saravanan, N.; Ramachandran, K.I. Fault diagnosis of spur bevel gear box using discrete wavelet features and decision tree classification. Expert Syst. Appl. 2009, 36, 95649573. [Google Scholar] [CrossRef] [4] Mark, W.D.; Lee, H.; Patrick, R.; Coker, J.D. A simple frequency- domain algorithm for early detection of damaged gear teeth. Mech. Syst. Signal Process. 2010, 24, 28072823. [Google Scholar] [CrossRef] [5] Hayes, M. Better Bevel Gear Production. Available online: http://www.gearsolutions.com/article/detail/6076/better-bevel-gear production (accessed on 23 August 2016).

[6] Golnabi, H.; Asadpour, A. Design and application of industrial machine vision systems. Robot. Comput. Integr. Manuf. 2007, 23, 630637. [Google Scholar] [CrossRef] [7] Sun, T.H.; Tseng, C.C.; Chen, M.S. Electric contacts inspection using machine vision. Image Vis. Comput. 2010, 28, 890901. [Google Scholar] [CrossRef] [8] Cubero, S.; Aleixos, N.; Molto, E.; Gomez-Sanchis, J.; Blasco, J. Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables. Food Bioprocess Technol. 2011, 4, 487504. [Google Scholar] [CrossRef].

[9] Fernandes, A.O.; Moreira, L.F.E.; Mata, J.M. Machine vision applications and development aspects. In Proceedings of the 2011 9th IEEE International Conference on Control and Automation (ICCA), Santiago, Chile, 1921 December 2011.

[10] Carrio, A.; Sampedro, C.; Sanchez-Lopez, J.L.; Pimienta, M.; Campoy,

P. Automated low-cost smartphone-based lateral flow saliva test reader for drugs-of-abuse detection. Sensors 2015, 15, 2956929593. [Google Scholar] [CrossRef] [PubMed] [11] Huang, K.Y.; Ye, Y.T. A novel machine vision system for the inspection of micro-spray nozzle. Sensors 2015, 15, 1532615338. [Google Scholar] [CrossRef] [PubMed] [12] Liu, Z.; Li, X.J.; Li, F.J.; Wei, X.G.; Zhang, G.J. Fast and flexible movable vision measurement for the surface of a large-sized object. Sensors 2015, 15, 46434657. [Google Scholar] [CrossRef] [PubMed] [13] Fremont, V.; Bui, M.T.; Boukerroui, D.; Letort, P. Vision-based people detection system for heavy machine applications. Sensors 2016, 16,

128. [Google Scholar] [CrossRef] [PubMed] [14] Perez, L.; Rodriguez, I.; Rodriguez, N.; Usamentiaga, R.; Garcia, D.F. Robot guidance using machine vision techniques in industrial environments: A comparative review. Sensors 2016, 16, 335. [Google Scholar] [CrossRef] [PubMed] [15] Liu, R.; Zhong, D.; Han, J. Bevel gear detection system with multi- camera vision. Hsi-An Chiao Tung Ta Hsueh/J. Xi'an Jiaotong Univ. 2014, 48, 17. [Google Scholar] [16] Qiao, N.; Ye, Y.; Huang, Y.; Liu, L.; Wang, Y. Method of circle detection in pcb optics image based on improved point hough transform. In Proceedings of the 4th International Symposium on Advanced Optical Manufacturing and Testing Technologies: Optical Test and Measurement Technology and Equipment, Chengdu, China, 19 November 2008.