Predicting stock prices has always been a complex and challenging task due to market volatility and the influence of diverse factors. In this blog, we introduce a novel Transformer-based approach for stock price prediction, integrating multimodal data such as stock prices, trading volumes, and financial sentiment analysis.
Highlights
- Leverages the Transformer encoder architecture for time series modeling.
- Incorporates multimodal features, including sentiment analysis using FinBERT.
- Achieves satisfactory performance on key metrics like RMSE, MAE, and MAPE.
Introduction
Stock price prediction has long been a critical focus for investors and researchers. The chaotic nature of the stock market, driven by factors such as macroeconomic trends, geopolitical events, and company fundamentals, makes accurate forecasting difficult. Our model, StockFormer, addresses these challenges by integrating multimodal data to capture complex relationships and dependencies.
Datasets Overview
Our experiments utilized data from three major stocks: Apple (AAPL), Google (GOOG), and Amazon (AMZN).
Model Architecture
The StockFormer architecture builds upon the Transformer encoder framework, incorporating the following components:
- Input Features: Combines stock prices, trading volumes, and sentiment scores extracted using FinBERT.
- Positional Encoding: Uses sinusoidal encoding for efficient sequence alignment.
- Multi-head Self Attention: Captures both short-term and long-term dependencies in the data.
- Feed Forward Network: Enhances the learning capacity and generalization of the model.
Experimental Results
We conducted extensive experiments to evaluate the effectiveness of StockFormer on multimodal data. Below are the key metrics:
Ablation Study
To understand the impact of each feature on the model's performance, we conducted an ablation study. The results demonstrated that incorporating multimodal data (price, volume, and sentiment) significantly improved prediction accuracy.
Visualization
Below is the visualization comparing predicted and actual Amazon stock prices.
Conclusion
StockFormer reveals the potential of Transformer models in financial data analysis. By integrating multimodal features, it provides a robust framework for stock price prediction, with applications extending to trend prediction and market analysis.
Future work includes exploring additional features such as macroeconomic indicators and company fundamentals, as well as adapting the model for classification tasks like predicting stock movement directions.
