Illusion : A Smart Music Player

Illusion : A Smart Music Player

Ravi Kumar1, Numer Mulla2, Dishank Gangar3

Department of Computer Engineering, Shah and Anchor Kutchhi Engineering College Mahavir Education Trust Chowk, W.T Patil Marg, Chembur, Mumbai, Maharashtra 400088

Abstract Now a days music players are one of those applications that is always present in any users mobile phone. For most users, listening to music is a distressing and feel good activity. For many, listening to music is a fun activity, many use it to just relax, many use it for gym, exercise and running to boost their energy levels. Also, it is proven that listening to right kind of music at right time can shoot human emotions by changing human psychology and hence can make humans feel better and more relaxed. It was observed that most of the music players in India just recommend songs based on the users behaviour or similarity between songs. It becomes annoying whenever the user has to change the playlist and current music a number of times, which most probably does not suit the users current mood, current activity, type of song user wants to listen at a particular moment of time, based on the environment, time, activity, etc. So, Illusion is a smart music player application, that will detect any users current heart rate data and his/her current activity status and thus music player will figure out the list of songs that can be played to better suit the user at any moment of time, automatically.

Keywords Accelerometer Data, Music Player, Heart Rate, Emotion, Music Recommendation

1. INTRODUCTION

   Most of the people around the world uses music player. Also, the demand for better music player is rising day by day, which is one of the booming markets in every country. Unfortunately, most of the products of music player applications recommend songs based on the users preference given explicitly or similarity between different musics. Many a times this makes the user to change the current music track a number of times to suit his/her current emotional status, activity status and environment, which can create irritation for any user. Illusion, is a smart music player that recommend songs automatically based on the users current heart rate and the current activity level that the user is performing at any particular moment of time. Also, feedback and behaviour of the user for every music can be used to recommend music that will be more precise based on the conditions of the user and the music. Apart from this the objective of the smart music player is to enhance the mood of the user from a bad state where the user could be in stressed out, anger, crying, feeling bad, depressed state etc., to a more good state where the user feels good and hence become happy, just with the help of some tweaks in users psychology unknowingly, which will be done by our application.

2. RELATED WORK

   One of the works was the creation of music recommendation system that recommends music based on the user behaviour. This work was conducted by students of computer science at University of Miami, USA [1]. Their goal was to recommend song that would favour the user based on

   the act that the user take on the current music played on the music player. Also, it checks for the freshness of songs so that songs feel fresh to users ear every time.The advantages of this system is that the approach which is used in this paper considered genre, recording year, freshness, favor and time pattern as factors to recommend songs. The evaluation results demonstrate that the approach is effective. The disadvantages of this system is that it is not considering any human inputs like heartbeat or motion data which proves to be useful for detecting human emotions. With the use of such factors accuracy gradually increases.

   Another work includes the creation of music player which recognizes the user emotion based on just the voice of the user and recommend songs accordingly. The advantages of this system is that the it is using voice based song recommendation approach and has an overall accuracy of 81.57% although the disadvantages could be that it could have used more inputs for detecting emotions instead of just voice signals i.e. heart rate data and accelerometer data. The work was done by students of Electronics and Telecommunication Department of Fr. CRIT, Mumbai, India [2].

   Next work was given by the students of Information Technology, King Mongkuts Institute of Technology, Bangkok, Thailand [3]. The work was Emotion Based Music Player. Here the music was recommended by the music player solely based on the emotions that was felt by user and the prediction of users current emotional state was taken by the users heart rate and by facial recognition.The advantages of this system is that the experimental results present that detecting the happy emotion is the most precise with around 98%. The disadvantages of this system are that emotion detection based on facial image recognition is susceptible to many disturbances and facial expression analysis suffers from face oval partial cover, location of the camera, lower quality, insufficient or uneven illumination. The accuracy results proves to be indefinitely low as the accuracy of the sad mood detection is the lowest with 40%. The song database has lesser song data too.

   Two more similar work was done by students from US and professors from Germany and South Korea [4][5], where the music was recommended by the music player to the user based on the activity that the user was performing at any moment of time. The advantages of this system is that it has good accuracy results. The deep residual bidirectional gated recurrent neural network (DRBGRNN) with data augmentation has the best activity recognition rate of 98.4%The disadvantages of this system is that it is only recommending music based on accelerometer data which detects user activity whereas emotions play an important role too.

3. PROPOSED SYSTEM ARCHITECTURE

   1. System Requirements

      • Android 5.0 or above.

      • Mobile Accelerometer.

      • Smart Band.

      • RAM 1 GB and above.

   2. System Design and Implementation

   1. System Overview

      The system overview of the proposed application is presented in Fig. 1. The mobile application will collect users heart rate from smart band connected to application via blue- tooth, then the application will collect accelerometer data from mobile accelerometer so as to detect users activity level. The data will be pushed to server, processed and then the application will suggest songs that will suit users current status from the database where, even the music is classified into categories to match the users activity level.

      Fig. 1 System Overview

   2. Working Model

      This application requires hardwares such as mobile accelerometer and smart band. The smart band is required to be connected to the android application using bluetooth. The application is then used to collect the heart rate data from smart band and then the accelerometer data from mobile accelerometer. The data thus collected is then pushed to the server for the necessary processing. On the server side, based on the heart rate data and accelerometer data collected from the application, it is then processed to find the activity level of the user. Then based on heart rate and activity level of user, a playlist is being generated from the database that suits and matches the heart rate as well as activity level of the user, based on research studies. The database consist of approximately 850 songs. Each songs in the databas has been classified into three music tempo category based on the tempo of the overall music. The activity of the user is determined by the accelerometer data obtained from mobile accelerometer.

      Fig. 2 Working Model

   3. Implementation

   GUI Design : The GUI basically is a music player system that is designed based on android application. It will contain basic features like play button, next button, previous button as well as activity button that will start collecting the users accelerometer data and heart rate data from accelerometer device and smart band respectively. Music will be played using the GUI play bar.

   Database Creation : The database consist of various types of songs consisting of Hindi, English, lyrics based as well as instrumental based songs. Our database contains music files in

   .mp3 as well as .ogg file formats. The main component of the database will be the music data file, which will contain information regarding path at which all music files are stored, music names and tempo associated with each music files. These data then later be used by the application server to predict relevant music files that needs to be given to the user, based on their current heart rate as well as current activity level.

   Extraction Of Data : Relevant data that needs to be fetched before any processing starts, will be implemented in this module. Two very important data sets that need to be fetched are accelerometer data and heart rate data. Accelerometer data consist of body/linear accelerometer data (x-, y-, z- coordinates), gravity accelerometer data (x-, y-, z- coordinates) and gyroscope data (x-, y-, z- coordinates). Heart rate data will consist of a number which will denote the current heart rate of the user, at any moment of time.

   Model Creation : Model for predicting users activity level needs to be build. This model thus then be used to predict user activity level based on the data retrieved from android

   application. The data received will be processed first to extract relevant information out of the raw data and thus further will be applied to this model to predict users activity level. Deep neural network can be used to create the model. Model thus

   created by our approach gives us an accuracy of 95.91% while training and this is then saved for later use.

   Activity Prediction : The accelerometer data (that is linear accelerometer data, gravity accelerometer data and gyroscope data) that is received from android application at the server end is then processed to create a excel file. This excel file is then used by our program/algorithm to generate 261 features out of the 9 received features. These new features are then applied to the activity prediction model created earlier to predict the users activity level. The users activity level is classified into three categories that is low, moderate and high activity level. The activity level thus generated after processing is then stored in a text file for further processing.

   Final Implementation : In this phase various other small tasks will be achieved, that will be remaining for proper completion of the project. Some of these tasks could be establishing connection between front end application and the back end server, collecting as well as sending relevant data to and from mobile application to server, playing music files, creating apis for different buttons such as play button, next button, previous button etc.

4. RESULTS AND ANALYSIS

   The GUI was designed and tested successfully. Application consisted of four buttons where the functionalities of buttons were to get heart rate data and accelerometer data, base URL field area where user can specify the servers IP address, first song button which will send all the extracted data to the server and thus will receive the first song and the last button fetches the next songs within the playlist created at server. Database for music tempo was generated and was stored in a text file for further processing. The file consisted of path of the database and details of music such as name of music file and its tempo. Extraction of certain data like accelerometer data (that is linear accelerometer, gravity accelerometer and gyroscope) and heart rate data was performed successfully and was displayed temporarily for analysis purpose. The model for predicting users activity level was created successfully, attaining an accuracy of 95.91%. Then the testing of model was performed by taking 9 features of accelerometer data and calculating 261 features out of it. These features were then given to the model and the activity status was predicted and was stored in a text file. Heart rate was categorized into 3 levels that is low, normal and high. Data from android application was received successfully and the results are stored in text file for further processing. All the testing and results were observed and final implementation of overall system was performed and thus the music was played accordingly.

5. CONCLUSION

We propose a smart music player which will be implemented as an Android application. This application aims to suggest songs based on emotion of the user at any moment, activity of the user at that moment, preference set by that user and previous behaviour of the user based on songs, timing, circumstances etc. The main functions of the application would be finding users current mood, users current activity, users current explicit preference that is sensitivity of song preferred by user and based on the users implicit preference that is users past

behaviour which includes active play times, passive play times, skip time, forgetting curve etc. for songs, recommend relevant songs which would be relevant for the user at that moment of time. Also, the main objective of the application is to improve users mood with the help of the concept of Pattern Break in Psychology.

ACKNOWLEDGEMENT

We would like to thank our guide Mr. Atul Kachare, Head of the Computer Engineering Department for always providing us with their valuable guidance and suggestions. We would also like to thank the panel of examiners for their valuable input during all the stages of the presentations and for their time and effort they put to assess our work. Without the help and assistance of the people mentioned above the successful completion of the project would not have been possible. Through the course and whole of our life, I would thank my family who has always supported me in my growth.

REFERENCES

1. Nextone Player: A Music Recommendation System Based on User Behavior. 12th International Society for Music Information Retrieval Conference (ISMIR) 2011.

2. Music Player Based on Emotion Recognition of Voice Signals. International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT) 2017.

3. Emotion Based Music Player. King Mongkus Institute of Technology Ladkrabang, Bangkok, Thailand.

4. XPOD A Human Activity and Emotion Aware Mobile Music Player. University of Maryland Baltimore County, United States.

5. Music Recommendation System Using Human Activity Recognition from Accelerometer Data. IEEE Transactions on Consumer Electronics 2019.

6. Accelerometer Data Preparation for Activity Recognition. Department of Intelligent Systems, Jozef Stefan Institute, Jamova, Ljubljana, Slovenia.

7. National Centre for Biotechnology Information https://www.ncbi.nlm.nih.gov/

8. Learn Making Music – https://learningmusic.ableton.com/

9. Machine Learning Tutorials – https://www.youtube.com/playlist?list=PLu0W_9lII9ai6fAMHp- acBmJONT7Y4BSG

10. Keras Tutorial Tensorflow – https://www.youtube.com/watch?v=Y1- hQdgftMQ

11. Python Network Programming – https://www.youtube.com/playlist?list=PLhTjy8cBISErYuLZUvVOYs R1giva2payF

12. Python Documentation – https://docs.python.org/3/

13. Flask Documentation – https://flask.palletsprojects.com/e/1.1.x/

14. Android Documentation – https://developer.android.com/docs

15. Librosa Documentataion – https://librosa.github.io/librosa/

16. Scientific Python Documentation – https://www.scipy.org/docs.html

17. UCI Machine Learning Repository – https://archive.ics.uci.edu/ml/datasets/human+activity+recognition+usi ng+smartphones

18. Bluetooth Low Energy API – https://developer.android.com/guide/topics/connectivity/bluetooth-le

19. Motion Sensors – https://developer.android.com/guide/topics/sensors/sensors_motion

20. Relationship between heart rate and preference for tempo of music, Hiroshima University.

21. Relationship between exercise heart rate and music tempo preference, research quarterly for exercise and sports, American alliance for health, physical education, research and dance, 2006.

22. Resting heart rate : what is normal ? DN research, Swiss national science foundation.

23. Preferred tempo reconsideration, IPEM department of musicology, Ghent University, Belgium.

24. Cardiovascular, cerebrovascular and respiratory changes induced by different types of music in musicians and non-musicians : the importance of silence, University di Pavia, Italy.

25. Music and the heart Circulation, Journal of The American Heart Association.

26. Statistics Groov in the heart : Matching beats per minutes to heart rate.

27. Article Lets go physical : The psychology of effective workout music.

28. Jerk signals –

    https://physics.stackexchange.com/questions/431595/how-to-calculate- the-jerk-from-acceleration-data

29. FFT – https://math.stackexchange.com/questions/30464/what-is-the- difference-between-the-discrete-fourier-transform-and-the-fast-fourier- transform