Statistical Assessment of Accuracy of the KinectTM as Preparation for Develop of a Work Measurement Tool

Statistical Assessment of Accuracy of the KinectTM as Preparation for Develop of a Work Measurement Tool

Martin Gonzalez Moncada

Department of Graduate Studies and Research, Instituto Tecnológico Superior de Nuevo Casas Grandes

Nuevo Casas Grandes, Chih., México.

Daniel Gerardo Bencomo Trejo

Department of Graduate Studies and Research, Instituto Tecnológico Superior de Nuevo Casas Grandes

Nuevo Casas Grandes, Chih., México.

Abstract An experimental study was performed in order to determine the accuracy for measurements obtained by KinectTM and establish operating conditions to improve the results, to achieve this purpose some statistical tests were made for dimensions measured with this device. This project was carried out under the following conditions, persons of different body size included, three different light conditions, five different distances between the subject and device, and device was placed at five different heights with respect to the work surface for each one distance. With this study was achieved statistically validate the capabilities of Kinect TM, and its performance level, to develop activities related to the measurement of work and establish its range and operating conditions.

Keywords Kinect, Statistical Assessment, Operating Conditions, Skeletal Tracking

1. INTRODUCTION

   According to Pagliari and Pinto [1], in recent years, the videogame industry has been characterized by a great boost in gesture recognition and motion tracking. The Microsoft Kinect sensor allows acquiring RGB, IR and depth images with a high frame rate. Because of the complementary nature of the information provided, it has proved an attractive resource for researchers with very different backgrounds.

   KinectTM is a motion sensor and it provides a Natural User Interface (NUI) that allow users to interact without any intermediary device, such as controller. This device was released in late 2010 by its manufacturer [2]. It is considered as a revolutionary device, mainly in the areas of computer vision [3], because it opened many new topics for research as robotic [4-6], biomedical engineering [7] and measurement system [8-10] as well as the interactive possibilities it enables [11]. Its use drastically increased in last years, because of its accessibility, affordability, and usability. Even though, this device is not widely spread in countries that are under development, worldwide companies are using it for their own technology development. Step aside, shortly after the Kinect launch, the device was hacked by third party communities, and Software Development Kits (SDK) were created by them

   Jorge de la Riva Rodríguez

   Department of Graduate Studies and Research, Instituto Tecnológico de Ciudad Juárez Ciudad Juárez, Chih., México.

   Rosa María Reyes Martinez

   Department of Graduate Studies and Research, Instituto Tecnológico de Ciudad Juárez Ciudad Juárez, Chih., México.

   and have spread throughout the Web [1], Microsoft subsequently released, on February 2012, the first official SDK for Windows [12], permitting the sensor to be used not only as a game device, but also as a measurement system [13], because its software API provides the real-time position of the body joints of each user (). Although its main function was for playing Xbox video games, the KinectTM Sensor currently offers a new prospect for the development and application of affordable, portable and easy-to-use markerless motion capture technology [14].

   The SDK enables users to develop sophisticated computer- based human motion tracking applications in C# and C++ programming languages. The immersive KinectTM technology from both hardware design and the SDK makes it possible to detect, track and recognize human motion dynamically in real time. Applications of KinectTM have been extended to many fields beyond video games, including healthcare, education, retail, training, virtual reality, robotics, sign languages and other areas. Moreover, researchers have intensively studied fundamental techniques for human motion tracking and analysis using KinectTM [12].

   KinectTM camera and its skeletal tracking capabilities have been embraced by many researchers and commercial developers in various applications of real-time human movement analysis, however currently accuracy of this device is still unknown [15]. Human motion tracking is widely used for movement analysis. Movement analysis has numerous applications, security, biomechanical analysis, medical diagnosis, serious or fun gaming.

   Currently, the majority of human motion tracking is performed using (MBS) Marker Based System technologies [16]. Blommestein, et al. [17] in its study, they had complications with using the camcorder, then to identify the extremities was necessary to use gloves bright colors, but this hindered the development of the task by the operator, hence they concluded that the MBS is not recommended. Other

   authors in their studies they emphasized to work the markerless motion tracking in human body extremities or joints [18].

   Around the world many researches were developed respect to calibration of KinectTM, focused mainly for skeletal tracking systems for specific tasks, but little attention has been paid to the evaluation of the accuracy of measurements obtained with KinectTM. As shown in these papers there are not enough evidence that work has been done respect to the measurement of work, supported by new technologies, such as computer vision devices. Our goal is to make a statistical assessment of accuracy of measurements obtained with this device, and prepare it to develop a useful tool for measurement of work, and the most important, will be non- contact tool with person or surface work. Statistical assessment allows us to establish some parameters, such as, confidence level of measures, operation range, and general conditions of operation, all them necessary to make reliable decisions.

   B. Method

   This was an experimental study in a laboratory, for its development static and dynamic measures was taken, both cases were conducted under the following conditions: three light intensities (lighted, semidarkness, and darkness).

   Besides, for static measurements was considered: five distances between the subject and device (1m to 3m at steps of 0.5m), and device was placed at four different heights (0.75m, 1.20m, 1.60m, and 2.00m) with respect to the work surface for each one distances as shown in Figure 2.

2. MATERIALS AND METHOD

   This section describes the materials used and the method under which the project was developed.

   A. Materials and Population

   A list of the materials used in this project are listed below: Microsoft Kinect Sensor camera was used as a markerless motion capture device. The Microsoft Kinect SDK software and Microsoft Visual C#, both installed on a laptop (Intel Corei7 CPU T7200@2.2GHz, 16Gb of RAM, a USB 3.0 port, and Windows 10). The Kinect camera does not require any calibration; whenever a subject is in the cameras field of view his/her stick figure skeleton is automatically created. The measurement frequency of the Kinect sensor is approximately 30Hz (frequency varies slightly during operation). It was also necessary to use a camera tripod, Kinect to USB adapter, a set of medium Lego size pieces, and a flexometer.

   With respect to the participants, persons with different body size were included in this project, that was classified into four types: thin teenagers, normal size, overweight, and obese (Fig. 1).

   Figure 1: Body size of persons considered in this project

   Figure 2: Heights for snsor and distances between subject and device

   For testing, it was necessary to draw a scale on the floor, with the five distances set on a tape as indicates white arrow in Figure 3.

   Figure 3: Scale on the floor

   To take the measures the subject holding a flexometer placing the scale marks at center of the palm of your hand (See Fig. 1). The opening the arms was six different distances between both hands, and the central vision line of Kinect at center of hands as shown in Figure 4.

   Figure 4: Different distances between hands that will be measured

   Twelve measurements were taken for each one distance.

   Fig 5 Show the screen capture software

   Figure. 5 Screenshot of the software

   For dynamic measurements, furthermore environmental conditions, were made tests on surface area, taking dynamic measurements in real time. Firstly, were identified three areas on work surface as shown in Figure 6, each of them of 10 cm of diameter (approximately a fist size) The task consisted of assembling (the task is to assembly) two pieces taken from points B and C and assemble them while they are held, and then place them at point A. This activity was developed by persons with different body size, too.

   Distance between each one of the points are 40 centimeters, this measure was taken with device while the operator developed the task

   Figure 6: Identification of areas on work surface for dynamic measurements

   Also, a test was done for 4 points, three containers and one assembly area, as shown in Fig. 7. Both yellow containers had only pieces of Lego, each one of the

   Respect to the illumination, three light intensities (lighted, semidarkness, and darkness) were tested, in all cases the body joints were detected without problems, just in some cases, several objects (Fig. 8) with shape similar to that of the human body (specifically with water jugs and chairs) were detected as people, and that caused errors in the readings, due to the overlap of the joints of two bodies. This happened with the three different light insights.

   Figure. 8 Objects with shape similar to that of the human body

3. ANALYSIS AND RESULTS

   Twelve measurements were taken for each distance and each height, being in total twenty readings. For each of one couple the distances between subject and device and heights for sensor, the following statistical tests were performed.

   1. Linearity and bias test

      Was made a linearity and bias test for each one of distances (0.20, 0.30, 0.60, 0.80, 1.00, and 1.20 meters) between hands, each one heights for sensor (0.75m, 1.20m, 1.60m, and 2.00m), and the subject place at different distances from device (1m to 3m at steps of 0.5m). Figure 9 shown results of linearity test the device on the surface, one meter between subject and device, and all distances (0.20, 0.30, 0.60, 0.80, 1.00, and 1.20 meters) between hands. In this test, it can be appreciated all P-values 0.05, except for 0.20m between hands, the reason for this is that the subject was very close to the sensor and with the arms extremely open, as a result the hands were sometimes outside the vision area of the device. The scatter plot shows, and confirm, that the points for 0.20 m between hands, at the same time that allows us to appreciate that the more hands are together, the system of measurement is inaccurate

      Estudio del sesgo y de la linealidad del sistema de medición para Medicion1

      N otificado por: M artin G onzalez

      containers, as well as the assembly area, were separated

      N ombre del sistema de medición : M edicion a 1 metros de D istancia F echa del estudio:

      T olerancia: M isc:

      by 40 cm from center to center.

      4

      3

      2

      Se sgo

      1

      0 0

      -1

      Regresión IC 95%

      Datos

      Sesgo prom.

      Linealidad del sistema de medición

      P redictor C oef E E del C oef. P C onstante 0.3446 0.2256 0.131

      P endiente 0.001956 0.002896 0.502

      S 0.839409 R-cuad. 0.6%

      S esgo del sistema de medición Referencia S esgo P P romedio 0.481528 0.000

      20 0.631667 0.000

      40 0.317500 0.027

      60 0.358333 0.020

      80 0.251667 0.046

      100 0.531667 0.078

      120 0.798333 0.125

      -2

      20 40

      60 80

      100

      120

      Figure 7: Taking dynamic measurements for an assembly process of pieces of Lego

      The task is to take a piece of each yellow container, take them to the assembly area (white area), once assembled they are placed in the blue container.

      V alor de r efe r encia

      Figure 9: Linearity test for the device on the surface, and one meter between subject and device

      Similar results, for the other distances, are shown in Figure 10

      Figure 10: Similar results for other distances

   2. R & R Test

      Figure 11 shown results for R&R test for the device on the surface, two meters between subject and device, as shown the graph (Fig. 10 a), all variance is between parts, this information is confirmed in the table (Fig 10 b), P-value = 0 for parts, and P-value > 0.05 for operator and interaction between both (operator and parts). Similar results occur for all distances from 1.5 meters to 3 meters

      Figure 11: R&R test for the device on the surface, two meters between subject and device

      For distances less than 1.5 meters, measurements were inaccurate, the arms were very open and the hands went out of the range of vision of the Kinect Sensor.

   3. Establishing operating parameters

   With the tests performed, it was possible to establish and validate the operation range of the Kinect Sensor. Figure 12 shows the adequate range of operation so that the measurements obtained with this device are reliable. The area marked in green, shows the range in which the sensor must operate, while in the area marked in red, sensor presents a great variation. the white lines (Horizontal y Vertical) represent the central line of vision, which indicates a vertical viewing angle of 27º above and below the central line, and an angle of 21.5º on each side of the horizontal line of sight. n Figure 12 we can clearly see that the closer the subject of the sensor is, the greater the possibility of it leaving the viewing angle of the device.

   Figure 12: Kinect Sensor operation parameters

4. CONCLUSIONS

This device works perfectly as a measurement tool, to obtain both static and dynamic measurements, as long as the subject is between 1.5 meters y 3 meters, respect the Kinect Sensor.

Kinect sensor have capability to acquire measurements in any light condition, with the same precision.

In distances from zero to less than 1.5 meters the Kinect Sensor presents inconsistency in the readings, but in distances greater than 1.5 meters, its measurements are efficient.

The body size does not interfere with the result of the reading made by the Kinect sensor.

In short, this device is efficient provided that the distance between the subject and the sesnor is superior to 1.5 meters, regardless of the lighting conditions, the complexion of the person and that there are no objects with a shape similar to that of the human body near the person

ACKNOWLEDGMENT

The preferred spelling of the word acknowledgment in America is without an e after the g. Avoid the stilted expression one of us (R. B. G.) thanks …. Instead, try R.

B. G. thanks….

REFERENCES

1. D. Pagliari and L. Pinto, "Calibration of Kinect for Xbox One and Comparison between the Two Generations of Microsoft Sensors," Sensors 2015. doi: 10.3390/s151127569 Available: www.mdpi.com/journal/sensors

2. T. Dutz, S. Hardy, M. Knöll, S. Göbel, and R. Steinmetz, "User interfaces of mobile exergames," in Human-Computer Interaction: Applications and Services, 2014.

3. W. Zhao, "A concise tutorial on human motion tracking and recognition with Microsoft Kinect," SCIENCE CHINA Information Sciences, vol. 59, Accessed on:August 23, 2016. doi: 10.1007/s11432-016-5604-y Available: info.scichina.com link.springer.com

4. A. Oliver, S. Kang, B. C. Wunsche, and B. MacDonald, "Using the Kinect as a Navigation Sensor for Mobile Robotics,"

5. J. Suarez and R. R. Murphy, "Using the Kinect for Search and Rescue Robotics " 2015.

   Available: https://www.researchgate.net/publication/236160253

6. R. Macknojia, A. Chavez-Aragon, P. Payeur, and R. Laganiere, "Calibration of a network of Kinect sensors for robotic inspection over a large workspace," 2016.

   Available: https://www.researchgate.net/publication/261339881

7. M. Alnowamia, B. Alnwaimib, F. Tahavoria, M. Coplanda, and K. Wellsa, "A Quantitative Assessment of using the Kinect for Xbox360TM For Respiratory Surface Motion Tracking," in Proceedings of SPIE – The International Society for Optical Engineering 2012.

8. K. Khoshelham, "ACCURACY ANALYSIS OF KINECT DEPTH DATA," in International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences,, Calgary, Canada, 2011, vol. XXXVIII.

9. J. C. K. Chow and D. D. Lichti, "Photogrammetric Bundle Adjustment With Self-Calibration of the PrimeSense 3D Camera Technology: Microsoft Kinect." doi: 10.1109/ACCESS.2013.2271860

10. M. Kenneth David and R. Tess Alethea, "The Kinect: A low-cost, high-resolution, short- range 3D camera," in EARTH SURFACE PROCESSES AND LANDFORMS, 2012: Wiley Online Library.

11. S. Izadi et al., "KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera," UIST11,

12. R. Lun and W. Zhao, "A Survey of Applications and Human Motion Recognition with Microsoft Kinect," International Journal of Pattern Recognition and Arti c ial Intelligence, vol. 29, no. 5, p. 48, 2015.

13. D. C. Guevara, G. Vietri, M. Prabakar, and J.-H. Kim, "Robotic Exoskeleton System Controlled by Kinect and Haptic Sensors for Physical Therapy," presented at the Biomedical Engineering Conference, 2013.

14. X. Xu, R. W. McGorrya, L.-S. Choub, J.-h. Linc, and C.-c. Changd, "Accuracy of the Microsoft KinectTM for measuring gait parameters during treadmill walking," Gait & posture, 2015.

15. B. Bonnechere et al., "What are the current limits of the KinectTM sensor?," Laval, France,, 2012: Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies.

16. A. Cappozzo, U. D. Croce, A. Leardini, and L. Chiari, "Human movement analysis using stereophotogrammetry: Part 1: Theoretical background," Gait & Posture, vol. 21, p. 11, 2005.

17. D. L. v. Blommestein, A. F. v. d. Merwe, S.Matope, and A. D. Swart, "AUTOMATION OF WORK STUDIES: AN EVALUATION OF METHODS FOR A COMPUTER BASED SYSTEM," CIE42 Proceedings, pp. 112-1 to 112-14, 2012.

18. A. L. Brooks and A. Czarowicz, "Markerless motion tracking: MS Kinect and Organic Motion OpenStage," Proc. 9th Intl Conf. Disability, Virtual Reality & Associated Technologies, p. 3