This repository contains the dissertation and associated code for "A Novel Approach to Stock Trading with Group-Agent Deep Reinforcement Learning," submitted as part of my Masters Degree at the University of Manchester in 2024.
This project investigates the application of Group-Agent Reinforcement Learning (GARL) to stock market trading. It addresses a gap in current Reinforcement Learning (RL) research applications to algorithmic trading by exploring GARL's potential to improve trading performance and maximize returns. The study also compares GARL-based strategies against traditional machine learning (ARIMA) and standard single-agent RL algorithms.
-
Environment Design: Created a custom RL environment using OpenAI Gym Framework specifically for stock trading simulation.
-
Strategy Design & Implementation: Designed, implemented, and fine-tuned a trading strategy based on the GARL model.
-
Baselines for Benchmarking: Developed two baseline strategies: one using traditional machine learning (ARIMA) and another using standard single-agent Deep Reinforcement Learning (DQN, PPO, A2C).
-
Data: Utilized daily stock price data from large-cap US companies (Google, Apple, Microsoft) on the Nasdaq-100 index (2001-2020 for training, 2021-2023 for testing). Enhanced data with 20 popular technical indicators for improved model learning.
-
Backtesting: Simulated all developed strategies on historical data within a realistic trading environment.
-
Performance Analysis: Analyzed, interpreted, and evaluated the performance and returns of the GARL approach against the baseline strategies using key financial metrics.
| Strategy | Cumulative Return (MSFT) | CAGR % (MSFT) | Sharpe Ratio (MSFT) | Max Drawdown (MSFT) |
|---|---|---|---|---|
| GARL | 105.09% | 18.12% | 1.03 | -32.49% |
| Single-Agent DQN | 98.27% | 17.33% | 1.27 | -17.7% |
| Online ARIMA | 333.35% | 66.71% | 2.89 | -11.05% |
| Buy and Hold | 77.21% | 14.18% | 0.83 | -37.15% |
| Single-Agent PPO | 45.66% | 9.11% | 0.68 | -21.59% |
| Single-Agent A2C | -45.82% | -13.24% | -0.75 | -57.4% |
Key Findings:
-
The Online ARIMA strategy demonstrated the most robust performance across all metrics, significantly outperforming both RL and buy-and-hold strategies.
-
GARL showed higher returns and CAGR than single-agent DQN, but at the cost of increased risk (lower Sharpe Ratio, higher Max Drawdown). It allows for distinct learning pathways among agents.
-
Single-Agent DQN was the most effective among the RL algorithms, showing good adaptability and risk management.
-
PPO was conservative, preserving capital during downturns but generally underperforming buy-and-hold.
-
A2C performed poorly, indicating unsuitability for this task.
-
The limited stock market data size was identified as a major challenge for deep RL algorithms, potentially leading to overfitting.