DeepSeekMath Meets Order Book: Group-Aware Policy Optimization for High-Frequency Directional Trading
Researchers applied group-aware reinforcement learning methods, Group Relative Policy Optimization (GRPO) and Group Sequence Policy Optimization (GSPO), to high-frequency trading using limit order book data. These techniques, inspired by large language model training, outperformed traditional methods like Q-learning and standard PPO in profitability, returns, and risk control on stocks such as AMZN, AAPL, and GOOG. Using order-flow imbalance features improved trading signals. However, results are limited to one hour of data per asset, so broader validation is needed.
What it examines
This paper explores using reinforcement learning, specifically group-aware policy optimization methods like GRPO and GSPO, for high-frequency directional trading on limit order books. It aims to improve trading performance by extracting order-flow imbalance features and compares these new methods to traditional approaches like Q-learning and PPO.
What it concludes
Group-aware policy optimization methods showed better trading results and lower risk than standard methods in this study. These techniques could help build smarter trading agents for financial markets. However, more research is needed to test them in different market conditions and with more realistic trading costs and scenarios.
Evidence objects
Researchers applied group-aware reinforcement learning methods, GRPO and GSPO, to high-frequency trading with limit order book data, outperforming traditional Q-learning and PPO in profitability, average returns, and risk control for major stocks.
key_findings bullet 1 · key_findings · validation V0
A standout innovation is using order-flow imbalance (OFI) features as a compact state representation, enabling deep neural networks to extract meaningful trading signals from complex market data with surprising effectiveness.
key_findings bullet 2 · key_findings · validation V0
Group-normalized policy updates led to substantial improvements in returns and downside risk, but results are limited by testing on just one hour of data per asset, making findings promising yet not conclusive.
key_findings bullet 3 · key_findings · validation V0
This paper uniquely adapts DeepSeekMath-inspired group-aware policy optimization (GRPO, GSPO) from LLM training to high-frequency trading, leveraging limit order book data and order-flow imbalance. Its methodological innovation demonstrates superior empirical performance over standard RL baselines, offering compelling cross-domain novelty, though experimental scope limits immediate impact and paradigm-shifting significance.
key_findings bullet 4 · key_findings · validation V0
Raw abstract and provenance
- … This paper studies reinforcement learning for high-frequency trading on limit order books by pairing an Order-Flow-based state model with policy-gradient methods. …
Source row: 571 · abstract type: snippet