Face Recognition Classification in Proctored Exam

Face Recognition Classification in Proctored Exam

1st Akshat Shrivastava

Computer Science and Engineering MIT ADT University

Pune, India

2nd Ashray Parmar

Computer Science and Engineering MIT ADT University

Pune, India

3rd Anurag Sen

Computer Science and Engineering MIT ADT University

Pune, India

4th Ayush Soun

Computer Science and Engineering MIT ADT University

Pune, India

5th Prof. Suresh Kapare Computer Science and Engineering MIT ADT University

Pune,India

AbstractWith the advent of COVID-19, remote learning has blossomed. Schools and universities may have been shut down but they switched to applications like Microsoft Teams to finish their academic years. However, there has been no solution to examinations. Some have changed it to an assignment form where students can just copy and paste from the internet, while some have just cancelled them outright. If the way we are living is to be the new norm there needs to be some solution. We shall aim to track the eyeballs of the test-taker and report if he is looking to the left, right, or up which he might do to have a glance at a notebook or signal to someone. This can be done using Dlibs facial key point detector and OpenCV for further image processing. We used the pre-trained weights of YOLOv3 trained on the COCO dataset to detect people and mobile phones in the webcam feed.

Index TermsFace Recognition, Exam, Gaze Tracking , Person Counting, Mobile Counting

1. INTRODUCTION

   The online education system, which increasingly demands full remote teaching, continues to find its Achilles heel in the evaluation system. Nowadays, according the UNESCO Educational Disruption and Response to COVID-19 crisis, most governments around the world are closing the educational institutions and moving their activity to online and remote modality impacting over 89student population. For example, in Spain, most universities have decided to move the exams in online modality. Thus, e-proctoring tools are very pertinent and significant supporting this process. The e-proctoring (electronic proctoring) is a system formed by electronic tools that allows the monitoring of the remote evaluative process through telematic resources, trying to make the results reliable . [1] Despite the fact that there are already electronic proctoring tools that seek to guarantee the quality of the evaluation process without requiring the physical presence of the student in a specific place or the union of the student and the examiner in that place, e-proctoring still continues without widespread use

   in institutions. [2] Although a change in trend has been taking place with the appearance of massive

   Identify applicable funding agency here. If none, delete this.

   online open courses (MOOCs) is in open education in general and in online education in particular where e-proctoring is increasingly useful. Therefore, and all together for this total distant educating to be stretched out to foundations that dont yet have faith in the achievement of its usage, this examination centers around finding the persuasive factors deciding the execution of this assessment framework and tries to advance its utilization through these. For this, a bibliographic report is completed that permits a rundown of the most powerful inspirational components to be uncovered while tolerating the utilization of new mechanical instruments (that is, the point at which the instructive framework acknowledges this apparatus as a technique for far off oversight) and a causal guide of the framework is created with them, in which specialists in internet instructing measure and break down the most compelling elements when it comes to acknowledgment by instructive establishments. [3]

   In information assortment (for the development of the causal guide), the meeting is utilized as an instrument, a causal guide is made for every one of the interviewees, and a worldwide causal guide is thusly made with the normal evaluation of the specialists, which permits its examination through the FCMappers instrument, which demonstrates the most persuasive variables, the components that get the most impact from the rest, and the most pertinent components inside the framework. In the accompanying area, electronic delegating instruments in instructive organizations are contextualized through an abstract survey featuring the circumstance of oversight in distant educating, e-delegating in the instructive framework is contextualized, and the most compelling

   inspirational elements in tolerating the utilization of new innovative instruments are point by point. We proceed with the approach utilized in this examination furthermore, the examination of the outcomes and close by calling attention to the inspirational elements deciding the usage of e-administering in internet instructing [10] [9]

2. RELATED WORK

   When preparing measures are done in far off spots, the presence of understudies and educators or analysts who confirm their insight turns into an issue.

   In this unique circumstance, attempting to maintain a strategic distance from this presence, a large portion of the advancements and changes have happened in the educating learning measures, leaving the assessment framework consigned to actual presence in a particular spot, for reasons of oversight. In any case, a point has been arrived at where the instructive framework and society request devices that try to guarantee the nature of the assessment cycle without fundamentally requiring actual presence in a particular spot.

   This interest has prompted the development of devices that permit observing this distant assessment framework through telematic assets , accomplishing e-administering in web based instructing. [1]

   The e-administering is being utilized consistently in MOOCs, where online courses attempted to force themselves as substitute items for conventional educating however discovered their fundamental issue in the management of their evaluative tests for the resulting accreditation of the information accomplished. As of today (since 2017), stages, for example, edX (one of the fundamental stages in MOOCs) as of now utilize this approach

   The IEEEtran class file is used to format your paper and style the text. All margins, column widths, line spaces, and text fonts are prescribed; please do not alter them. You may note peculiarities. For example, the head margin measures proportionately more than is customary. This measurement and others are deliberate, using specifications that anticipate your paper as one part of the entire proceedings, and not as an independent document. Please do not revise any of the current designations. [4]

   This administering technique permits:

   Visual and hear-able reconnaissance, utilizing the sound and cameras of the examinees PC or cell phone, just as observing the PC, if the test is mechanized (since it permits the alternative of taking electronic or paper-based tests). There are now numerous PC instruments that permit seeing the PC screen where the understudy steps through the exam and, also, there are uncommon programs for e-delegating that keep the understudy from leaving the test screen and just taking the test without talking with some other application or program on that PC. Frameworks, for example, Remote

   Delegate NOW (RPNOW), eProctoring, SMOWL or ProctorExams, utilize their own frameworks to guarantee control of the PC on which the understudy plays out the test. [5] This suggests that the online instructive framework is significantly preferred by its appearance, by permitting its assesment cycle to be completed distantly and the actual

   presence of the understudy in a particular place not being needed. Be that as it may, its focal points are excluded from this by itself, since the utilization of this apparatus additionally permits the versatility of the tests to characterize the degree of the understudy on a scale, for example, that of the European System of Reference for Languages or in Adaptive Tests

   This apparatus has an ever increasing number of uses, in fields as assorted as in online staff enlistment, where it is utilized increasingly regular , and in virtual grounds, where it is developing dramatically . [6] [6]

   Similarly, its utilization is filling extraordinarily in geologically scattered domains and in arising nations (offering open doors for complete far off preparing) , in new types of instructing enlarged reality in designing investigations , in offbeat conditions inside the instructing learning measures ,in new types of schooling, for example, MOOCs or microdegrees (giving believability in the ID while assessing and confirming information) , and in the assessment and affirmation of capabilities to rehearse callings (as is done at the International Authorizing Body and at Indiana University ).

   E-delegating will wind up winning in web based educating, yet the facts confirm that there are issues at the point when the tests are not solid and steady to be completed under this methodology or when vigilantes or inspectors dont have the foggiest idea the proper behavior when issues emerge with respect to the response of the understudy or issues in telematic correspondence. This has prompted the improvement of numerical models and calculations to improve the conduct of the individuals who plan the tests when taking them and of those who screen these tests , in light of a legitimate concern for improving the e-administering framework.

   Notwithstanding the previously mentioned data and that ProctorU (one of the most widely recognized Proctoring frameworks right now) reported in 2017 that it had just completed in excess of 4 million tests by this methodology , a large number of the web based encouraging establishments dont actualize this apparatus in their assessment framework. Consequently, this examination looks to discover what are the inspirational elements answerable for this.

   To do this, a bibliographic report is first completed to make a rundown of the most compelling inspirational factors while tolerating the utilization of new innovative devices, looking through logical information bases and inspecting those logical articles that connected factors and persuasive elements with the utilization of innovative devices, (for example, the article by Sanchez-Prieto, Huang, Olmos-Miguel´ a´nez, Garc ´a- Penalvo and Teo ),

   lastly gathering the dissemination that the creator R. Arteaga makes in her examination after an broad scholarly audit by specialists in this field . [6] [7]

3. PROPOSED METHOD

The AI will have four vision-based capabilities which are combined using multithreading so that they can work together:

Gaze tracking Mouth open or close Person Counting Mobile phone detection

Apart from this, the speech from the microphone will be recorded, converted to text, and will also be compared to the

text of the question paper to report the number of common words spoken by the test-taker.

1. REQUIREMENTS

   • OpenCV

   • Dlib

   • TensorFlow

   • Speech recognition

   • PyAudio

   • NLTK

2. GAZE TRACKING

   We shall aim to track the eyeballs of the test-taker and report if he is looking to the left, right, or up which he might do to have a glance at a notebook or signal to someone. This can be done using Dlibs facial keypoint detector and OpenCV for further image processing. I have already written an article on how to do real-time eye-tracking which explains in detail the methods used that will be used later on. This is very similar to eye detection. Dlibs facial keypoints are again used for this task and the test-taker is required to sit straight (as he would in the test) and the distance between the lips keypoints (5 outer pairs and 3 inner pairs) is noted for 100 frames and averaged. If the user opens his/her mouth the distances between the points increases and if the increase in distance is more than a certain value for at least three outer pairs and two inner pairs then infringement is reported.

3. PERSON COUNTING AND MOBILE PHONE DETECTION

   I used the pre-trained weights of YOLOv3 trained on the COCO dataset to detect people and mobile phones in the webcam feed. For an in-depth explanation on how to use YOLOv3 in TensorFlow2 and to perform people counting If the count is not equal to an alarm can be raised. The index of mobile phones in the COCO dataset is 67 so we need to check if any class index is equal to that then we can report a mobile phone as well. [3] [9]

4. COMBINING USING MUTLITHREADING

   As eye-tracking and mouth detection are based on dlib we can create a single thread for them and another thread can be used for the YOLOv3 tasks: people counting and mobile detection. First, we import all the necessary libraries and along with the helper functions. Then the dlib and YOLO models are loaded. Now in the eyes mouth() function, we find out the facial key-points and work on them. For mouth detection, the original distances between in the outer and inner points are already defined and we calculate the current ones. If a certain amount is greater than the predefined ones, then the proctor is notified. For the eyes part, we find out their centroids as shown in the article linked and then we check which facial keypoints are they closest to. If both of them are on the sides then it is reported accordingly. In the count people and phone() function, YOLOv3 is applied to the webcam feed. Then the classes of objects detected are checked and appropriate action is taken if more than one person is detected or a mobile phone is detected. These functions are passed to in separate threads and have infinite loops in them which the proctor can break by pressing q twice.

5. AUDIO

   The idea is to record audio from the microphone and convert it to text using Googles speech recognition API. The API needs a continuous voice from the microphone which is not plausible so the audio is recorded in chunks such there is no compulsory space requirement in using this method (a ten-second wave file had a size of 1.5 Mb so a three-hour exam should have roughly

   1.6 Gb). A different thread is used to call the API so that a continuous recording can without interruptions, and the API processes the last one stored, appends its data to a text file, and then deletes it to save space. After that using NLTK, we remove the stopwords from it. The question paper (in text format) is taken whose stopwords are also removed and their contents are compared. We assume if someone wants to cheat, they will speak something from the question paper. Finally, the common words along with its frequency are presented to the proctor. The proctor can also look at the text file which has all the words spoken by the candidate during the exam. [1] [8]

   Until line 85 in the code, we are continuously recording, converting, and storing text data in a file. The function read audio(), as its name suggests, is used to record audio using a stream passed on to it by stream audio(). The function convert() uses the API to convert it to text and appends it to a file test.txt along with a blank space. This part will run for the entire duration of the examination. After this, using NLTK, we convert the text stored to tokens and remove the stop-words. The same is done for a text file of the question paper as well and then common words are found out and reported to the proctor.

   RESULTS

6. Vision

   • Track eyeballs and report if candidate is looking left, right or up.

   Find if the candidate opens his mouth by recording the distance between lips at starting.

   • Instance segmentation to count number of people and report if no one or more than one person detected.

   • Find and report any instances of mobile phones.

   • Head pose estimation to find where the person is looking.

   • Face spoofing detection

7. Face Detection

   It is implemented in face detector.py and is used for tracking eyes, mouth opening detection, head pose estimation, and face spoofing.

   An additional quantized model is also added for face detector as described in Issue 14. This can be used by setting the parameter quantized as True when calling the get face detector(). On quick testing of face detector on my laptop the normal version gave 17.5 FPS while the quantized version gave

   19.5 FPS. This would be especially useful when deploying on edge devices due to it being uint8 quantized.

8. Facial Landmarks

   Earlier, Dlibs facial landmarks model was used but it did not give good results when face was at an angle. Now, a model

   provided in this repository is used. A comparison between them and the reason for choosing the new Tensorflow based model is shown in this article.

   It is implemented in face landmarks.py and is used for tracking eyes, mouth opening detection, and head pose estimation.

9. Eye and mouth tracking eye tracker.py is to track eyes. mouth opening detector.py is used to check if the candidate opens his/her mouth during the exam after recording it initially. Its explanation can be found in the main article, however, it is using dlib which can be easily changed to the new models.

10. Audio

    It is divided into two parts:

    Audio from the microphone is recording and converted to text using Googles speech recognition API. A different thread is used to call the API such that the recording portion is not disturbed a lot, which processes the last one, appends its data to a text file and deletes it. NLTK we remove the stopwods from that file. The question paper (in txt format) is taken whose stopwords are also removed and their contents are compared. Finally, the common words along with its number are presented to the proctor.

11. Detection face spoofing.py is used for finding whether the face is real or a photograph or image. head pose estimation.py is used for finding where the head is facing. [10]

CONCLUSION

This paper presents a sight and sound examination framework for on the web test administering, which intends to keep up scholastic respectability in e-learning. The framework is moderate and helpful to utilize from the content takers point of view, since it just needs to have two cheap cameras and a mouthpiece. With the caught recordings and sound, we separate low-level highlights from six fundamental parts: client confirmation, text location, discourse discovery, dynamic window recognition, look assessment and telephone location. These highlights are then prepared in a transient window to procure significant level highlights, and afterward are utilized for cheat discovery. At long last, with the gathered information base of 24 test takers speaking to certifiable practices in online test, we exhibit the capacities of the framework, with almost 87Web based instructing still doesnt offer total distant educating as a rule, since there are numerous foundations that, in the assessment cycle, keep on requiring the actual presence of the understudy in a explicit spot to join the understudy and the inspector in said place, for administrative reasons. Nonetheless, there are as of now e-administering apparatuses that permit this cycle to be completed distantly, without requiring that actual presence. Also, there is an ideal pattern in the use of this technique in MOOCs and in open training around the world. [5]

Along these lines, this investigation has tried to find the inspirational elements deciding the execution of this assessment framework, permitting the piece of a rundown of persuasive affecting variables when tolerating the utilization of new mechanical apparatuses (that is, the point at which the

instructive framework acknowledges this device as a strategy for distant management), and figure out which are the most compelling or unequivocal when it comes to acknowledgment by instructive organizations. The rundown is comprised of the accompanying inspirational components: Quality administration (QM), accessible data (AI), outside molding (EC), trust (T), saw similarity (PC), saw helpfulness (PU), mentality (A) and aim (I). The most conclusive factor in this cycle is trust (T), which would be the level of security and protection that establishments have in the utilization of this device (e-delegating). This agrees with the fundamental line of examination on this apparatus, where a large portion of the investigations center around trust also, the wellbeing of utilizing it. Something that can be found investigations from years back, as can be seen in Howlett and Hewett , is that both mechanical arrangements and instructional plan answers for diminish cheating in these far off assessments were at that point inspected. Be that as it may, this has not changed, and right up til the present time the exploration proceeds a similar way regardless of the great working of the models that were appeared in the scholarly audit and that reviews for example, those of Atoum, Chen, Liu, Hsu, and Liu , Davis, Rand, and Seay , OReilly and Creagh , and Kayser , stress the trust and security in edelegating, and prescribe its utilization to online instructive foundations (and even instead of vis-a-vis). This implies that` the merchants of these kinds of apparatuses should complete great correspondence whats more, publicizing effort with respect to the security and protection of this instrument in the event that they need to overcome whats more, solidify its utilization in web based educating organizations. Not just that, they should likewise support and boost the remainder of the persuasive variables since the trust factor gets the most impact from the remainder of the elements. [1]

Therefore, in this mission, they should likewise offer satisfactory earlier data to these organizations, since many dont know about its attributes (accessible data), the advantages of its utilization (saw helpfulness), similarity with its sort of business (saw similarity), and viability and quality with obvious outcomes (quality administration). Furthermore, they should demand their customers to publicize their utilization of this instrument, with the goal that they can fill in as an illustration to other foundations (outside molding). The other two variables (disposition and aim), being two of the three factors generally subject to the rest, additionally owe their significance inside the framework and their degree of effect on different components. Subsequently, as a rule terms, it very well may be said that this is planned to change the vision of establishments devoted to web based educating with respect to e-delegating and to energize the utilization of this apparatus in their foundations, permitting total distant instructing, which is the thing that the online instructive framework requests. This exploration is restricted to the investigation of persuasive elements in the usage of e-delegating in instructive foundations, yet this impediment can be killed in future investigations and expand this examination to different kinds of components that additionally impact this execution, for example, the investigation of the shortcomings of the distinctive innovative instruments utilized by this far off oversight framework. [3]

REFERENCES

1. Helaine M Alessio, Nancy Malay, Karsten Maurer, A John Bailer, and Beth Rubin. Examining the effect of proctoring on online test scores. Online Learning, 21(1):146161, 2017.

2. Mohammad Dadashzadeh. The online examination dilemma: to proctor or not to proctor?

3. Pillip Dawson. Defending assessment security in a digital world: preventing e-cheating and supporting academic integrity in higher education. Routledge, 2020.

4. AI Maria Joseph and B Asoke Nath. Integration of massive open online education (mooc) system with in-classroom interaction and assessment and accreditation: An extensive report from a pilot study. In Proceedings of the international conference on e-learning, e-business, enterprise information systems, and e-Government (EEE), page 105. The Steering Committee of The World Congress in Computer Science, Computer …, 2013.

5. Rebecca Kantar, Keith McNulty, Erica L Snow, Richard Wainess, Sonia D Doshi, Devon BWalker, and Matthew A Emery. The rise of immersive cognitive assessments: Towards simulation-based assessment for evaluating applicants. In International Conference on Artificial Intelligence in Education, pages 438441. Springer, 2018.

6. Thomas Langenfeld. Internet-based proctored assessment: Security and fairness issues. Educational Measurement: Issues and Practice, 39(3):2427, 2020.

7. Teoh Ai Ping. Students interaction in the online learning management systems: A comparative study of undergraduate and postgraduate courses. Asian Association of Open Universities Journal, 2011.

8. C Raj Kumar and Anand Prakash Mishra. Why indian law schools need to adopt the online law entrance exam in 2020: The r (e)-imagination of the law school admissions process. 2020.

9. Prakash Sinha and Aman Yadav. Remote proctored theory and objective online examination. Int. J. Advanced Networking and Applications, 11(06):44944500, 2020.

10. Ludwig Slusky. Cybersecurity of online proctoring systems. Journal of International Technology and Information Management, 29(1):5683, 2020.