Remote Doctor : Tele Medicine Unit

Remote Doctor : Tele Medicine Unit

D. G. K. Lakshani, W. K. S. De Silva, U. R. R. I. S. Bandara,

1. W. H. V. Samarasinghe, K. A. D. C. P. Kahandawaarachchi and K. N. Pulasinghe

   Faculty of Computing

   Sri Lanka Institute of Information Technology, Malabe, Sri Lanka

   Abstract Remote Doctor is a solution for transfiguration of telecommunication for healthcare industry to address the prominent aspects identified among the rural hospitals in Sri Lanka such as inadequate specialist consulting, poor medical record storage, poor capacity planning, time consuming prescription writing and report viewing. The proposed telemedicine system inscribes with video conferencing and report sharing among the physician and the patient, automated prescription, taking snapshots and video clips of necessary details such as wounds, ulcers and incision, annotate them and storing them by compressing without damaging the quality of the images and videos. The system has been developed using PHP, JavaScript, Html5, quickBlox API, Web Speech API, WebRTC technology and Sphinx toolkit. The Remote Doctor will be utilizeed in government hospitals to communicate with rural communities.

   KeywordsTele Medicine; Remote Medicine; Video Conferencing; Data Compresion; Wound Annotation; Voice to Text Conferencing; Electronic Prescription

   1. INTRODUCTION

      The requirement for telemedicine is to consult a physician when the patient cannot reach a medical center. For a better telemedicine system, it should be capable with high quality video conferencing, better bandwidth, portable, cost effective, reachable and user friendly. The telecommunication technologies facilitate the ambulances, intensive care units, laboratories and pharmacies to communicate as one system. However, in Sri Lanka the existing infrastructure is either obsolete or the transmission capacity is inadequate for telemedicine which is a major challenge [1].

   2. BACKGROUND CONTEXT

      Telemedicine is a combination of telecommunication and information technologies to provide clinical health care at a distance to save lives in emergency situations [2]. It allows physicians to evaluate, diagnose and treat patients in remote locations using telecommunication and also allows patients to access medical expertise immediately, efficiently and without traveling [3]. Remote doctor is a system which is developed in a cost effective way to provide those facilities and it will help to reduce the gap of specialty knowledge and the technology between rural hospitals and city hospitals.

      Remote Doctor, is created as a unit for the Health Information System (HIS) and is based on MVC architecture. This system is a combination of user interfaces which can be web based and a robust common framework for health care systems to facilitate interoperability, extensibility, high security and high performance. The trends observed in a telemedicine systems are as follows,

      TABLE I. BASIC NEEDS OF TELEMEDINCE

      Teleme dicine applica tions	Basic needs
      	Cost	Portability	Autonomy	Weight & Size	PC type	Camera quality	Communic ation needs
      Ambul ance	Medi um/ High	High	Hig h	High	Palmt op	Medi um/ High	GSM
      RHC	Medi um/ High	Low	Low	Low	Deskt op/ Lapto p	Medi um/ High	POTS, GSM
      Ship	Medi um/ High	Low/ Mediu m	Low /Me diu m	Low	Deskt op/ Lapto p	Medi um/ High	GSM, SATE LLITE
      Home care	Low	Low/ Mediu m	Low /Me diu m	Low	Deskt op/ Lapto p	High	POTS
      Intensi ve care room	Medi um/ High	Low	Low	Low	Deskt op	High	POTS, GSM

      According to the American Telemedicine Association, telemedicine saves the patients, providers and players money when compared with traditional approaches [4]. Further it describes how to use telemedicine to monitor chronic care patient or allowing specialist to provide care of patients. This clearly defines why the patients satisfaction is important in healthcare and what aspects a telecommunication system should think in developing such an application. There are several telemedicine systems which are popular among European countries such as American Tele Health, eVisits, eVito, MD Live, AW9 according to the above specification. But if they are to be used in Sri Lanka, there will be issues with the technological knowledge, techniques and cost. Therefore, the progress may have ceased by technological issues. Thus, as a solution Remote Doctor is developed to provide a low cost, user-friendly solution for the rural communities in Sri Lanka.

      There are several video conference units that has been developed for distance communication such as Adobe connect, Google Hangouts, Skype Premium, Cisco WebEx

      and Citrix GoToMeeting which cannot be used for Remote Doctor Application because the system should preserve for the bandwidth in rural areas which those technological advances systems do not support for. Also those system should be paid for the usage. Therefore it is not economically worthwhile for the use. Therefore the researchers used WebRTC Technology to develop the video conferencing and quickBlox API to do the resolution control.

      Video compressing techniques can be used in compact and robust way to store and transmission video data so it will be cost elective in terms of size, bandwidth and power consumption. ITU-T and ISO/IEC international standards are used in compressing video data. Encoder will compressed the video data to be stored or for to be transmitted. Decoder will decompress the video data after transmission or from the storage so the user can watch them.

      Fig. 1. System Overview

      The automated prescription is developed using voice to text conversion mechanism. Dragon NaturallySpeaking, CLIN1 Transcription, MModal Fluency Cloud Application, TalkChart are few voice conversion systems which is popular. Moreover, the accuracy level of the conversion is the most critical aspect of the automated prescription because it is a high risk function. The common voice recognition systems does not provide enough accuracy for the voice inputs of medical jargon. However, the researchers have used Sphinx Toolkit to develop the automated prescription for Remote Doctor with a medical dictionary.

      PictZar is a planimetry software programs that are used to take measurements on digital photographs. PictZar may be used to measure wounds in health care industry. This non- invasive tool is working with digital images, hence physician has to capture images using digital camera and need to download them to computer [5]. This process normally takes place via a USB connection, Hot Wire connection, compact flash card, SD media card or Bluetooth technology. Once downloaded, using the application physician can take the dimensions of the wound surface [6]. Pressure Ulcer Scale for Healing (PUSH) and PSST are another tool for monitor only the changes of a pressure ulcer over tie. PUSH tool assessed length x width, exudate amount, and tissue type of the pressure ulcer. PSST tool consists of fifteen scored and two non-scored items. The sored items assessed variables of wound size, depth, tissue characteristics and wound exudate, whereas non-scored items assessed wound location and shape [7]. To use these tools there should be trained staff or experienced wound nurses.

   3. RESEARCH OBJECTIVES

      Remote Doctor contributes to the growing needs of people with chronic and long-term conditions and to the preventative health agenda. This encourages people to consult by a doctor and have more chances to meet the doctor as the users desire. The evaluation, monitoring and analyzing all are done though the system and it is totally like meet up the doctor by visiting the doctors place.

      Remote Doctor provides open access to everyone and anyone who is willing to access service though the application login. Videoconferencing units are being placed throughout the areas in greatest need. The security and stability of this network presents any third party attacks which will harm to the users privacy. Through this application two partied can share the reports and other essential details with Hyper Text Transfer Protocol Secure (HTTPS). Furthermore, the consultants can take pictures during the communication and annotate them for future references. The storage is optimize by compressing data. Moreover, a Voice to Text Conversion is used in medical transcription into text when the doctor reads which reduces the time wastage which brings novelty to the application.

   4. RESEARCH METHODOLOGY

      In order to develop the proposed functionality Remote Doctor application, researchers identified four functional units which ware described below.

      1. Video Conferencing and Resolution Handling

         A WebRTC web application is a cluster of HTML and JavaScript which are acting reciprocally with web browsers. The WebRTC API allows to use and control the real-time browser function properly. The WebRTC web application also act mutually with the browser, using both WebRTC and other standardized APIs proactively and reactively. The API supports a broad set of functions, like connection management, encoding/decoding ability negotiation, selection and control, media control, firewall and NAT element traversal inside the video conferencing [8]. This API allows a JavaScript application to take advantage of the novel browsers real-time capabilities. The real-time browser function implemented in the browsers core provides and the functionality needed to establish the necessary audio, video, and data channels. The API was designed using three main concepts, namely; Media Stream which streams audio and video data, Peer Connection to communicate directly from browser to browser and Data Channel designed to exchange generic data in a bidirectional peer-to-peer fashion [9].

         During the video conversation there are several functions which can use to control the behavior of the conference such as, Take call, Accept call, Reject call, Stop call and those are implemented through the QuickBlox SDK which includes all function in WebRTC. First an account is created in QuickBlox SDK and it takes an application id and authorization key for reference. Thereafter, the application is configured according to user needs. Once configured, the system is enable to modify this platform according to the needs [10]. In Remote Doctor Web application QVGA, VGA, HD, DEFAULT buttons are added for control video resolution to during the video call which supports to bandwidth resolution control and that helps to get a clear picture of the video screen.

         Fig. 2. Video Conferencing

      2. Image/Video Compression

         This function is to take snap shots and video recordings during the video conferencing using JAVA. It creates an application that can capture the doctors screen and modified using giving dimensions which can only capture the previously defined video streaming area. So it will capture the doctors screen image which shows in the streaming video and doctor can use it to take images of patients wounds and other highlights which needs to be use in future. The video capturing records the doctors screen with the above settled dimension and the sound is taken through the mic and the speaker.

         Image compression is performed using Lossless Compression where the image size will be reduced. While the quality will be same as the original image which is captured in JPEG file type using PHP [11], [12]. In video recording the video is stored in the database without decreasing the quality and the standard of the video with the use of ffmpeg convert library [12].

         Fig. 3. Compression Mechanism

         The ffmpeg converts is a web based cross platform library that uses to convert the Remote Doctors audio and video to minimum size file types such as mkv or flv [13].

         Fig. 4. Screen recorder application and screen recording area

      3. Automated Medical Transcription

         Speech is known as a complex phenomenon which describes the sound sets using the individual master sounds [14]. Furthermore, speech is an audio stream which keep continuous flow in whether it is a stable state or in a dynamically changing state. Words are built by the phones but use of those phone sets may different from each other inside their speaking. Therefore a single word can be defined with more than one classes of sounds and phones [15]. It should clearly recognize the speech sound patterns to understand how is it produced and perceived. The speech is assembled with words and each word consists of phones. Automated Medical Transcription converts the prescription and the diagnosis into text when the doctor reads them. This

         reduces the paper work of the doctor. Initially this was implemented using the web speech API but, the accuracy level of the speech outcome is very low because the API does not support to most of the medical jargon. Furthermore, it does not support the grammar needed to specify in transcription. Therefore, a medical dictionary was implemented with voice recognition related to health industry.

         Fig. 5. Medical Transcription

         During this implementation the data preparation is the most critical path inside the training process. This database should reflect which kind of data is going to be trained during the acoustic model data preparation. It should be according to the schema of the research field. Therefore, the system uses fifty words from the medical jargon as a start. For training a database file and a wav folder is given which contains all the recorded voice clips. So the file structure of the database consists of a phonetic dictionary, phoneset file, language model, a list of fillers, a list of files for training, transcription for training, a list of files for testing and a transcription for testing. Using those resource files the training script runs to perform the training which uses sphinxbase-5prealpha, sphinxtrain-5prealpha and pocketsphinx-5prealpha packages [16]. After the training process the tool provides an html file including the compilation details during the training process and log files for each and every step during the training. The exact details of the decoding process such as alignment comparison with the transcription, speed and result of each file are proceed under the results unit which is generated during the training progress.

      4. Wound Annotation, Graphical Representation and Medical Report Transmission

      The basic methodology of the wound annotation is, select the wound surface that physician need to measure and get the basic dimensions of that selected area. After capturing the wound image, PHP code is implemented to upload that image to patient profile. When uploading that image, it is appeared in a canvas with a transparent square. For that cropper.js library was used called it when uploading the image. Then using the transparent square, margin of the wound is selected. Then the system provide the dimension of that selected area. If the physician need to save that selected wound surface, he can download that, using the particular buttons below the canvas. Also there are facilities like zoom image, rotate image, resize image. The purpose of this method is to identify

      the wound healing process of the patient. By measuring the surface area of the wound, even a small change in the wound can recognized by the physician.

      Fig. 6. Wound Annotation

      The basic methodology of graphical review is, record patients diagnose process in the graphs such as blood cholesterol level, blood sugar level etc. Here the system has only the line graphs because always patient data are varying with the time. So it is best to use line graphs [18]. Physician can add graphs to any patient and can record diagnose values there. So the patient or physician both can view the available graphs. PHP codes are implemented for adding new graphs, update existing graphs and view graphs. Further amcharts library was used to develop the graphs and that JavaScript is changed for retrieve values from the system database. When adding new graphs physician has to give patient name and graph name. Then those details saved in the database and when adding records to that, those patient names and graphs names are loaded into drop box. Physician has to give new records and submit. All those records will save in the database under the particular patient details. Today many patients are suffering from chronic wounds such as leg ulcers, pressure ulcers and diabetic ulcers. Those chronic wounds are longtime take to heal and therefore physicians have to maintain records of the wound longtime. So graphical review is a best way to convey those records using graphs.

      Fig. 7. Graphical representation of Data

      The basic methodology of medical report transmission is share medical reports with another party. If physician needs to send patients medical reports to another physician, in order to get further advices, then he can send it as an attachment. Also if patient needs to ask anything regarding his disease or need to send some medical records to

      physician, then patient can do it using this function easily. PHP code is implemented for send new attachments and view existing conversations. In the code, gave file type as any therefore through the system user can send any type of files like word documents, pdf, txt, jpeg, png etc. When loading the conversation to the system, PHP code was implemented. In that code, all the message ids are taken, then made them in descending order and the last message id is taken with its details.

   5. RESULTS AND DISCUSSION

      Video Conference in remote doctor web application develop using open source technology and it features can change according to satisfying the system users requirement which is of low cost for developing countries. Though this is implemented using open source technologies it can be easily use with anywhere in any device. Also the application can manage the resolution level of viewing capacity of the video according to the users needs. This is a solution for the internet with low bandwidth and this also ideal for the communication with rural areas. That also help to increase the quality of the system. Other than that opus audio codec and VP8 video codec help to increase video and audio quality of the video conference and that help to improve the voice to text conversion and video recording feature of this system.

      Taking snap shots using WebRTC technology is not supports http server connection [17]. Therefore the application has embedded with a new screen capturing mechanism using java which can control the area for recording y giving the X and Y dimension of the frame through an actionListner Function. That decide the height and the width of the frame which needed to be captured according to the video screen. The remote doctor application uses a lossless JPEG image compression using PHP. The doctor captures the JPEG images therefor JPEG image compression is suitable for the Remote Doctor System. This compression works when doctor uploads the images to the system. These high quality images requires lot of space to store them but when these images uploaded to the Remote Doctor System these images requires small space to store them and the image quality is same as the original images. This happen because of the PHP lossless compression that used in the system. When images uploading these images go through this compression and only the image size will be reduced. This helps to store lot of quality images in the system because the images size is low lot of images can be stored in the system. But Remote Doctor System is created using PHP codes. In PHP it doesnt allow to compress videos when they are uploading. So because of this matter videos wont be compressed when they are uploading to the system. So as a solution for this ffmpeg conversion can used to reduce the convert the videos. This ffmpeg conversion is used to convert videos to another video types that requires small space to store like mkv or flv. When converting the videos to smaller size video size like mkv or flv the quality wont be reduced only the size will be reduced. So Remote Doctor System space wont be fill easily because now the videos need small space to store therefor now doctor can store many videos in the system.

      When considering about the details which are outputted from sphinx training tool it gives the 77% of accuracy for the training. The accuracy of the training is basically based on the speaking accent ant the patterns of the speaker. At the beginning of the training the system should identified the Words to be trained and how they are created using the phones. Those process should be included in the dictionary file. So in the training it is comparing the sound patterns with the wav pattern of the spoken words and those words are described using the phones. The comparison can be done only when the tool has identified it as an identifiable name. If the sound wav pattern is not tally with the words described by the phone set, it gives an error message saying the words cannot be identifies. Further the words may not be identified correctly because of the noise in the environment. The system should filter the garbage noise which is come though the microphone when the speaking is identified. Here in this system it has been eliminated by training the tool with some set of noises such as cough, loud breath, throat clear, typing, singing etc. Therefor the system can identify those noises and eliminate them from the voice input. But to improve the accuracy this should be eliminated all the environmental noise that can be disturb to the system in a particular environment. According to the results generated though the VTC system using the Web Speech API the task is not completely success because it does not identify all the given names related to health industry. The libraries inside the API does not train for those tasks. Therefor the research switched into Sphinx tool kit which can train a speech recognition process from the beginning. There the research is success so far. But it has made an issue to gran the output given from the training tool to the Remote Doctor system. So when this compatibility issue is solved this may be the best solution for voice recognition processors in the health care industry.

      When considering about the output result of the wound annotation system, it gives 80% of accuracy. For test that, ten sample wound images are uploaded and annotated. Form those ten images, eight images were successfully annotated and uploaded to the system. So the accuracy is 80% with the wound annotation system. The accuracy will also basically base on the quality of the wound image, and the size of the wound that can see in the image. When capturing the wound, should capture closest and clear image of the wound. In that case when doing the capturing, it is better to provide a margin with the system. So a square is given and physician has to capture the wound within that margin. Bt wound image should not be small than the square, it should be within the margins. Further there is one more image is captured to find the place of the body that the wound is. Otherwise if physician captured only the wound, later he will not able to recognize where the wound is. Also the quality of the image is must to consider. In wound annotation process, wound is annotated using a captured image. So the quality of the image is must to consider. If it is not a much quality image, then margins of the wound will be unclear to the physician. Wound margin is very important because today most of the patients are suffering from chronic wounds like leg ulcers, pressure ulcers and diabetic ulcers. Those wound take long time to heal and there are very small changes happened with the time. When annotating and keeping record of those

      chronic wounds, wound image should be clearly captured. Otherwise system will give the wrong calculations for dimension and it will decrease the accuracy of the system.

      When considering about the output results of the graphical review and medical report transmission systems, it is more accurate and suitable for the Remote Doctor System. As earlier mentioned there are chronic wounds and they will take long time to heal. Not only the chronic wounds but also there are lifetime diseases like blood sugar, blood cholesterol etc. So when storing those data in graphs long time, there will be large no of data to convey. Further the data points of the graph will map closely and it will not be clear to recognize the small changes. Therefore zooming facility is added to the graphs and it will provide a more efficient facility to physicians. When transmitting files, user can transmit any type of files like pdf, word documents, notes, jpg or png images, etc. There is no any delay of transmitting files and the process is very furious as normal chat applications. So there is no need of training the medical staff because of its contiguity. Another thing is the security of the network. When doing medical report transmission, security of the network is very important. If it is not secured, then third party will access the patients privacy data.

   6. CONCLUSION

Tele communication is becoming more attractive topic among the healthcare sector these days. The more comfortable doctor visits and efficient service are added more demand to the service. Furthermore in doctors point of view they are also benefited from the system. The system reduce time wastage and helps to give more efficient and attractive services to the customers. Telemedicine has been cited as being particularly helpful in rural areas, where the shortage of healthcare providers had led to a lack of accessibility to both basic healthcare and specialty care. Also the pre and post- surgery activities also can handle successfully through the system. However there is a slight uncertainty when a telecommunication system is going to initiate in the industry. It takes some considerable amount of money for initiate those systems in hospitals. But mainlining is more efficient. Reliable and cost effective than others. But obviously if we do so this may become a best way as an alternative for the current health care system in Asian countries as they are not developed to provide same technology for all over the country for get in use for healthcare.

The Remote Doctor is contain with video conferencing system with high quality video resolution. Medical reports are transmit though the doctor and the patient or between the hospitals via the application which makes a huge easiness for both sending and receiving parties. The image capturing is enable with video clips and they can stored in the system with compressing them as a solution for the not enough storage issue, the companies struggling in these days. Also this has

automate the transcription writing and that reduces the doctors paper work into half. So as future work this may develop as android application which can use through mobile phones and other android devices. Also this can be included a scanner to identify the text though a device and read by the system itself. Further the system can improves with combining some medical equipment into it such as pressure meter, pulse meter, Blood sugar counter etc. These improvement may add more convenient into the Remote Doctor system as well as the users.

REFERENCES

1. "Model Policy for the appropriate use of telemedicinetechnologies in the practice of telemedicine," p. 10, 2014.

2. "Wikipedia," [Online]. Available: https://en.wikipedia.org/wiki/Telemedicine. [Accessed 27 February 2016].

3. A. G. telemedicine, "Telemedicine," AMD, 2016. [Online]. Available: http://www.amdtelemedicine.com/telemedicine- resources/telemedicine-defined.html.

4. "American Telemedicine Association," American Tele Medicine, 2016. [Online]. Available: http://www.americantelemed.org/.

5. "PictZar," [Online]. Available: http://www.podiatricultrasound.com/PictZar.htm#INTRODUCTION.

6. M. E. W. T. P. L. L. M. C. C. and O. M. , "Wound measured from digital photographs using photo digital planimetry software," Wounds, 2011.

7. P. H. M. M. T. J. P. P. S. S. and S. J. , "Assessment of wound healing," Wound Practise and Research, 2009.

8. S. l. &. S. P. Romano, Real Time Communication with WebRTC, 2014.

9. T. Levent-Levi, "How to send a file using webRTC data API," [Online]. Available: https://bloggeek.me/send-file-webrtc-data-api/.

10. "Quickblox," QUICKBLOX, 2016. [Online]. Available: https://quickblox.com/developers/Overview. [Accessed September 2016].

11. C. Hoffman, "What Lossless File Formats Are & Why You Shouldnt Convert Lossy to Lossless," 2015. [Online]. Available: http://www.howtogeek.com/142174/what-lossless-file-formats-are- why-you-shouldnt-convert-lossy-to-lossless/.

12. M. D. and F. K. , "Riversible integer to integer wavelet transforms for image compression".

13. John, "Create a video conference recorder using WebRTC, media devices and media recorder," 2015. [Online]. Available: http://forio.com/about/blog/create-video-conference-recorder-using- webrtc/.

14. A. Rudnicky, "Sphinx Knowledge Base Tool," [Online]. Available: http://www.speech.cs.cmu.edu/tools/lmtool-new.html.

15. "Tranining Acoustic Model for CMUSphinx," CMUSphinx, 2016.

16. Nuzhat Atiqua Nafis and Safaet Hossaim, "Speech To Text Convertion in real time," p. 7, 2015.

17. Z. "Taking still photos with WebRTC," 2015. [Online]. Available:https://developer.mozilla.org/en-US/docs/Web/ API/WebRTC_API /Taking_still_photos.

18. "Commonly used graphs," [Online]. Available: http://assessment.tki.org.nz/Using-evidence-for-learning/Reading-and- analysing-data/Topics/Common-graphs.