A Novel Hybrid Model Using Lstm and Rnn for Stock Market Prediction

A Novel Hybrid Model Using Lstm and Rnn for Stock Market Prediction

Mr. Sourav Dutta

Department of Computer Science and Engineering Netaji Subhash Engineering College

Kolkata, India

Mr. Apan Paul

Department of Computer Science and Engineering Netaji Subhash Engineering College

Kolkata, India

Mr. Sayan Samanta

Department of Computer Science and Engineering Netaji Subhash Engineering College

Kolkata, India

Mr. Sayan Ghosh

Department of Computer Science and Engineering Netaji Subhash Engineering College

Kolkata, India

Mr. Udatya Deb

Department of Computer Science and Engineering Netaji Subhash Engineering College

Kolkata, India

Dr. Chandra Das

Department of Computer Science and Engineering Netaji Subhash Engineering College

Kolkata, India

Dr. Shilpi Bose

Department of Computer Science and Engineering Netaji Subhash Engineering College

Kolkata, India

Abstract With the emergence of technological developments such as global digitalization, business forecasting has entered a technological era that has changed business models. With the rise of capital markets, the stock market has become an investment destination for many financial investors. Many analysts and researchers have developed tools and strategies to predict stock prices and help investors make better decisions. Advanced business models enable researchers to predict business using non- textual information from social platforms. The accuracy of predictions can be increased by using advanced learning deep learning techniques. Still researchers have been developing new methods to improve prediction accuracy. In this regard we have proposed a new hybrid predictive model to improve prediction accuracy of the stock price. From the experimental results it has been found that the proposed model shows superiority over the LSTM and RNN model on the datasets of SENSEX, NIFTY and MSFT(Microsoft).

Keywords Lstm, Rnn, Hybrid Model, Mse, Mae

1. INTRODUCTION

   Over the past few years, advances in information technology have changed the way business is done. The financial sector is one of the best and most effective products in the country's economy. The World Bank announced that the global economy exceeded $100 trillion in 2022 [10]. The stock market has been

   under the spotlight for the past few years, largely driven by advancements in technology. Investors look for tools and techniques that can increase profits and reduce risk. Financial forecasting has long been a concern for analysts in disciplines such as economics, mathematics, information science, and computer science. Profits from the stock market are important in predicting the stock market. Stock market, market capitalization, company performance, government policies, gross domestic product (GDP) of the country, inflation, nature of bankruptcy, etc. It depends on many factors such as. The concept of active trading explains that market value is often determined by new information and based on patterns and therefore cannot be predicted based on information. This was a common practice in the past. Researchers have found that market value is somewhat predictable based on technology. Historical business data can be analyzed and combined with information from social media platforms to predict changes in business and marketing. The effectiveness of job estimating often depends on the quality of the job for which it is used. Although researchers have used some techniques to improve product quality, more work needs to be done on the extraction and selection process.

2. LITERATURE SURVEY

   LSTM works higher than Gated Recurrent Unit on Non- Linear information[4] and also inventory involves big amount of Non- Linear statistics. Istiake Sunny et al. [4] brought forward a new framework for stock price prediction in which he engaged outstanding RNN models, LSTM and BI-LSTM, by means of altering the amount of epochs, dense layers, hidden layers, and lots of hidden layers elements utilized to discover a forecasting healthy that can be used to forecast future occurrences, both models can yield excessive accuracy with low RMSE on a easily accessible dataset from yahoo finance. The BI-LSTM version supplied less RMSE standards compared to LSTM however required greater time to compute[4].

   The trouble with stock market forecasting is that it's miles based totally on sizeable numbers[1], a large amount of statistics and are in large part reliant on the long-term past. As an end result, LSTM [1], [4] regulates blunders with the aid of helping RNNs by storing statistics for later use. The forecast will become an increasing number of correct as the level progresses[1]. for that reason, demonstrating that it's far far extra reliable than different methods.

   Ishita Parmar et al.[1] put forward a LSTM model in where LSTM layers had been stacked on every different component with the output cost of 256 and dropout ratio as 0.3 to increase the speed of schooling and avoid over fitting. The test rating of 0.00875 MSE changed into obtained for the stacked LSTM model [1].

   N. Sirimevan [5] has used the correlation among stock rate and sentiment to enhance the prediction. Twitter sentiments, internet information, historical inventory facts were used in conjunction with live stock fees from Yahoo Finance API. The sentiment rankings from twitter and net and the google fashion were given to the univariate and multivariate LSTM [5]. The models have been integrated thru Weighted Average Ensemble [5]. The accuracies reduced with the days. For first day the excellent prediction rating accuracy become 0.99, for seven days 0.92 and reduced to 0.62 until thirtieth day.

   LSTMs have been used by Nelson et al. [2] to forecast future inventory charge patterns utilizing stock rate and technical analysis statistics. The recommended LSTM model has shown extra precision than present fashions of system gaining knowledge of including random forests RF, multilayers perceptron MLP and pseudo- random models in experiments [2]. This model changed into created with a rolling window style in thoughts. On the end of each trading day, a fresh neural network was constructed, which means a sparkling set of weights become the usage of a brand new set of model schooling and justification facts. The version is trained the use of facts from the preceding 10 months of change preceding the contemporary, and the version's running is authenticated the use of statistics from the beyond week. The version become built with Google's TensorFlow, and it comprises of a sigmoid output layer that is fed with the aid of an LSTM input layer that takes both technical suggestions and stock valuing records as entry [2].Buyers fee is an ancient statistics as a basis for making investing choices[3]. In the attention layer, the model selects and learns input information by computing the weighting of the entire statistics, while the eye layer weights the function vector [3].

   For brief-term and long-time period prediction of stock marketplace, K.A. Althelaya et al. [6] evolved Stacked LSTM and Bidirectional LSTM. The writers used historical records from the Standard and P 500 Index (S&P500) [6]. The data was pre- processed and normalized. As a baseline, shallow NN and the fundamental form of LSTM have been utilized. For the sake of analyzing the system's performance, the ultimate fee become used. The authors presented two structures, one for short-time period forecast (in the future) and the other for lengthy-time period forecast (30 days) both having a ten-day window length. Both fashions had been assessed and differentiated against numerous fashions, specifically MLP- ANN and LSTM, with a view to verify their performance. In assessment to the opposite models, the BLSTM [6] had a decrease RMSE and MAE score. For both brief- and lengthy- time period forecasts, the BLSTM outperformed contemporary models and done more convergence [6].

   M. Nikou counseled an LSTM model and as compared it to the ANN, guide Vector Regression (SVR), and the RF models. The outcomes showed that the LSTM model achieved better than the alternative fashions in the take a look at in predicting the ultimate day stock expenses of iShares MSCI United Kingdom [7]. Sumeet Sarode [8] and associates engaged LSTM with the most recent trading data as inputs. information is accrued from a large set of commercial enterprise news for news evaluation. when the rate rises, the consequences are combined to make an offer. LSTM and its variants had been studied with the aid of Klaus Greff et al. [9] of their paintings. They studied 8 variants of LSTM. The gates are the main parts of LSTM [9].

3. PROPOSED WORK

   For this project the following methodologies are used:

   1. Recurrent Neural Network (RNN):

      Individuals do not continuously think from scratch. When you study these words, you will get each word the same way you caught on to the past words. You do not toss everything away and begin considering it from scratch. You are enthusiastic about your considerations. Typical neural systems cannot do this, which appears to be a drawback. For illustration, let's say you need to portray occasions that happen at each point in the motion picture. It is not clear how a typical neural arrangement employs thinking about past occasions in a motion picture to anticipate afterward occasions. Repetitive neural systems fathom this issue. They are systems with circles that guarantee the continuation of information. In Fig 1, portion A of the neural organization looks at a few inputs xt and yields the value ht. Circles permit messages to pass from one step of the organization to the next. These circles make neural systems appear a little more puzzling. In any case, if you think a little more, you will see that they are not much different from ordinary neural systems. A repetitive neural arrangement can be thought of as numerous duplicates of the same arrangement; each duplicate passes messages to its successors. Such a circumstance shows that the versatile number is not consistent with the framework and title. They are normal engineering for neural systems for this sort of data. But RNN, too, has its impediments. It's great if the "substance" is later so that it can be distributed legitimately. But RNN comes up short when it

      has to depend on inaccessible context (i.e., something learned a long time prior) to create the required objects. We can utilize recursive objects to diminish misfortune, but there are continuous issues with slope collapse and angle misfortune.

      Fig 1: Recurrent Neural Network

   2. Long Short Term Memory (LSTM) Network:

      Often referred to as "LSTM", this is a special type of RNN capable of long-term learning. It was proposed by Hochreiter and Schmidhuber (1997) and developed and extended by many in subsequent studies. They are very effective in solving many problems and are now widely used. LSTMs are specifically designed to avoid long-term dependency problems. Remembering information for a long time is actually their bad habit, not something they tried hard to learn! All recurrent neural networks have the form of a recurrent chain of neural networks. In an RNN model, this iteration would have a very simple structure, such as the tanh layer. LSTM is a recurrent neural network that processes sequences of data and learns to retain important information over time. New information learned by the network is added to the "memory", which is updated each time according to the importance of the new data to the model. Over the years, LSTMs have revolutionized speaking and writing, language understanding, prediction, and other applications that have become the norm today. A typical LSTM cell contains three gates, an input gate, an output gate, and a forget gate. These gates learn their weight and decide how much of the current information should be remembered and how much of previous learning should be forgotten. This simple model is an improvement over previous similar RNN models. As shown in the equation below, i, f, and o represent three gates: input, memory, and output. C is the case that holds the curriculum and provides the output. All these are calculated for each timestamp t, taking into account the data learned from timestamp (t-1). The memory gate decides how much information can be removed from the current state, while the input gate decides what to add to the current state. Try this. The output gate used in the last equation controls the output amplitude calculated from the first two gates. Therefore, unlike traditional feedforward neural networks, LSTMs can remember or delete portions of previous windows of data. His ability to read and train on windows (or time steps) of data makes his work unique. Let's create a model in Python. It has a LSTM chain structure. A normal RNN consists of one neural network layer. LSTM, on the other hand, has four interactive layers that enable excellent communication.

      Fig 2: Internal structure of LSTM

   3. LSTMs works in a three-step process:

      The first step in LSTM is to decide what data to extract from the cell at a particular time. It is determined with the help of the sigmoid function. It looks at the previous state (ht-1) and the current input xt and evaluates the function.

      • There are features within the second layer. The first is the sigmoid function, and the second one is the tanh characteristic. The sigmoid function comes to a decision which values to let through (0 or 1). The tanh characteristic offers the weightage to the values exceeded, determining their degree of importance from – 1 to 1.

        Step 3 is to determine what's going to be the final output. First, you want to run a sigmoid layer which determines what elements of the mobile country make it to the output. Then, you ought to place the cellular state through the tanh feature to push the values between -1 and 1 and multiply it by means of the output of the sigmoid gate.

   4. Adoption of Hybrid Model:

      In response to the underperformance of LSTM by itself, we decided to implement an hybrid model approach to enhance performance and potentially mitigate the shortcomings faced with LSTM. This model structured with three layers, integrating LSTM and RNN layers. This architecture is augmented with the utilization of hyper parameters, each carefully tuned to specific values. This meticulous adjustment of hyper parameters contributes significantly to the model's optimization. As a result of this comprehensive approach, the hybrid model demonstrates enhanced performance surpassing that of previous iterations. This outcome underscores the efficacy of employing a multi-layered hybrid framework coupled with strategic hyper parameter tuning in the pursuit of improved predictive accuracy within stock market analysis.

   5. Algorithm of the Hybrid Model:

   1. Data Preprocessing: First the data is cleaned. This involves tasks such as filling missing values, scaling features, encoding categorical variables, and any other necessary preprocessing steps to prepare the data for modeling.

   2. Input to LSTM Layer 1: Once the data is cleaned, it is fed into the first LSTM layer (LSTM Layer 1). The LSTM (Long Short-Term Memory) layer is a type of recurrent neural network (RNN) layer that is well-suited for sequence rediction tasks.

   3. Output of LSTM Layer 1 to RNN Layer: The output of LSTM Layer 1 is then passed to an RNN (Recurrent Neural Network) layer. In this case, a Simple RNN layer is used. RNN layers are designed to process sequences of data, making them suitable for tasks such as time series prediction.

   4. Output of RNN Layer to LSTM Layer 2: The output of the RNN layer is further passed to the second LSTM layer

      (LSTM Layer 2). This allows the model to capture more complex temporal dependencies in the data.

   5. Final Predicted Value: The output of LSTM Layer 2 is used to make the final prediction. This prediction represents the model's estimate for the target variable based on the input data and the learned patterns in the sequence.

   Fig 3 Block Diagram of the Proposed Hybrid Model

4. RESULT

   1. Dataset

      We decided to use the NIFTY 50 (2000-2022), SENSEX (2010-2024) market Index dataset and MSFT (2000-2023) Stock. NIFTY 50 is a benchmark Indian Stock Market Index that represents the weighted average of 50 largest Indian companies listed on the National Stock Exchange. The SENSEX, also known as the S&P BSE SENSEX, is the benchmark stock index of the Bombay Stock Exchange (BSE) in India. It consists of the 30 largest and most actively traded stocks listed on the BSE.

      Fig 4: MSFT Stock

      Fig 6: Sensex Stock

   2. Performance Metrics Used

      • Mean Squared Error (MSE): MSE measures the average squared difference between the actual and predicted values in a dataset. It squares the errors before averaging, which penalizes larger errors more than smaller ones. MSE is calculated using following given formula below:

        MSE =

        …(1)

        Where, n is the number of data points, is the actual value, is the predicted value.

        • Mean Absolute Error (MAE): MAE measures the average absolute difference between the actual and predicted values in a dataset. It does not square the errors, providing a more direct interpretation of the model's performance.

          MAE = (2)

          Fig 5: Nifty 50 Index

          Where, N is the number of data points, value, is the predicted value.

          is the actual

   3. Result on Dataset – Sensex (2010-2024)

      Model Name	Layers	Results
      		MSE (Mean Squared Error)	MAE (Mean Absolute Error)
      RNN	Single Layer RNN	0.037	0.157
      LSTM	Single Layer LSTM	0.153	0.145
      Hybrid Model**	(L+R+L)* ** Triple Layer	0.029	0.109

      *** L = LSTM layer, R = RNN Layer

      ** Hybrid Model represents LSTM + RNN + LSTM Layered Model.

      Fig 7: RNN SENSEX

      Fig 8: LSTM SENSEX

      Fig 9: Hybrid Model SENSEX

   4. Result on Dataset – Nifty-50 (2000-2022)

      Model Name	Layers	Results
      		MSE (Mean Squared Error)	MAE (Mean Absolute Error)
      RNN	Single Layer RNN	0.203	0.419
      LSTM	Single Layer LSTM	0.179	0.226
      Hybrid Model**	(L+R+L)* ** Triple Layer	0.121	0.128

      *** L = LSTM layer, R = RNN Layer

      ** Hybrid Model represents LSTM + RNN + LSTM Layered Model.

      Fig 10: RNN Nifty 50

      Fig 11: LSTM Nifty 50

      Fig 12: Hybrid Model Nifty 50

   5. Result Dataset – MSFT (2000-2023)

   Model Name	Layers	Results
   		MSE (Mean Squared Error)	MAE (Mean Absolute Error)
   RNN	Single Layer RNN	0.027	0.155
   LSTM	Single Layer LSTM	0.056	0.184
   Hybrid Model**	(L+R+L)* ** Triple Layer	0.022	0.088

   *** L = LSTM layer, R = RNN Layer

   ** Hybrid Model represents LSTM + RNN + LSTM Layered Model.

   Fig 13: RNN MSFT

   Fig 14: LSTM MSFT

   Fig 15: Hybrid Model MSFT

5. DISCUSSION AND CONCLUSION

In this study, we undertook the task of stock market prediction utilizing the NIFTY 50, SENSEX market index dataset, and MSFT stock data. NIFTY 50, representing the weighted

average of the 50 largest Indian companies listed on the National Stock Exchange, and SENSEX, the benchmark index of the Bombay Stock Exchange, provided rich datasets reflective of the Indian stock market's dynamics. Moreover, the inclusion of MSFT stock data enriched our analysis with insights from international markets.

Our approach was distinguished by the incorporation of factors previously overlooked in related works. We identified additional features believed to exert significant influence on NIFTY 50, SENSEX, and MSFT stock predictions, thereby enhancing the comprehensiveness of our predictive models.

In constructing our machine learning models, we meticulously considered four primary features: Date, Opening Price, Closing Price, and Volume. These features collectively formed a robust dataset pivotal for training our models.

Our analysis involved the utilization of recurrent neural network (RNN) and long short-term memory (LSTM) network models. While RNNs showcased their adeptness in handling sequential data, the theoretically promising LSTM models exhibited unexpected underperformance in practical applications. Consequently, we explored an hybrid modeling approach, integrating both LSTM and RNN layers, supported by meticulous hyper parameter tuning.

The hybrid model, comprising a three-layer architecture with LSTM and RNN layers, emerged as the most effective solution. This approach, enhanced by strategic hyper parameter tuning, surpassed the performance of previous models, as evidenced by decreased Mean Squared Error (MSE) and Mean Absolute Error (MAE) values across all datasets. The notable success of our hybrid model underscores the significance of leveraging multi-layered frameworks and rigorous hyper parameter optimization in stock market prediction endeavors.

Concluding everything, our study not only contributes to the refinement of predictive models for stock market analysis but also highlights the importance of adopting innovative approaches and comprehensive datasets for accurate and robust predictions in dynamic financial markets.

REFERENCES

1. I. Parmar et al., Stock Market Prediction Using Machine Learning, in First International Conference on Secure Cyber Computing and Communication (ICSCCC) , 2018, vol. ICSCCC 2018, pp. 574576. doi: 10.1109/ICSCCC.2018.8703332.

2. David M. Q. Nelson, Adriano C. M. Pereira, and Renato A. de Oliveira, Stock Markets Price Movement Prediction With LSTM Neural Networks, in International Joint Conference on Neural Networks (IJCNN), 2017, pp. 14191426. doi: 10.1109/IJCNN.2017.7966019.

3. D. Wei, Prediction of Stock Price Based on LSTMNeural Network, in Proceedings 2019 International Conference on Artificial Intelligence and Advanced Manufacturing, AIAM, Oct. 2019, pp. 544 547. doi: 10.1109/AIAM48774.2019.0113.

4. M. A. I. Sunny, M. M. S. Maswood, and A. G. Alharbi, Deep Learning- Based Stock Price Prediction Using LSTM and Bi- Directional LSTM Model, in 2nd Novel Intelligent and Leading Emerging Sciences Conference, Niles,Giza, Egypt, Oct. 2020, pp. 87 92. doi: 10.1109/NILES50944.2020.9257950.

5. N. Sirimevan, I. G. U. H. Mamalgaha, C. Jayasekara, Y. S. Mayuran, and C. Jayawardena, Stock Market Prediction Using Machine Learning Techniques, in International Conference on Advancements in Computing (ICAC),IEEE, vol. 8, 2019, pp.192197.doi:10.1109/ICAC49085.2019.9103381.

6. K. A. Althelaya, E. M. El-Alfy, and S. Mohammed, Evaluation of bidirectional LSTM for short-and long-term stock market prediction, in

   2018 9th International Conference on Information and Communication Systems (ICICS), 2018, pp. 151156. doi: 10.1109/IACS.2018.8355458.

7. M. Nikou, G. Mansourfar, and J. Bagherzadeh, Stock price prediction using DEEP learning algorithm and its comparison with machine learning algorithms, Intelligent Systems in Accounting, Finance and Management, vol. 26, no. 4, pp. 164174, Oct. 2019, doi: 10.1002/isaf.1459.

8. S. Sarode, H. G. Tolani, P. Kak, and L. CS, Stock Price Prediction Using Machine Learning Techniques, in International Conference on

   Intelligent Sustainable Systems (ICISS 2019), 2019, pp. 177181. doi: 10.1109/ISS1.2019.8907958.

9. K. G Greff, R. K. Srivastava, J. Koutnik, B. R. Steunebrink, and J. Schmidhuber, LSTM: A Search Space Odyssey, IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 10, pp. 2222 2232, Oct. 2017, doi: 10.1109/TNNLS.2016.2582924.

10. https://www.deccanherald.com/business/world-economy-headed-for- recession-in-2023-research-1175111.html