Deep Reinforcement Learning in Non-Markov Market-Making
This paper presents a deep reinforcement learning framework for optimal market-making using non-Markov jump-diffusion models with simulated and real data.
What it examines
This paper examines an optimal market-making problem in high-frequency trading using deep reinforcement learning. It employs non-Markovian price models—semi-Markov and Hawkes jump-diffusion processes—to simulate limit order book dynamics, offering a modern alternative to traditional stochastic optimal control approaches in volatile trading environments.
What it concludes
This study shows that deep reinforcement learning with the Soft Actor--Critic algorithm can enhance market-making by addressing non-Markovian dynamics and adverse fills. The robust strategies may be applied in optimal liquidation, statistical arbitrage, and other algorithmic trading areas, with future work aimed at improved order book modeling.
Evidence objects
Researchers unveil a novel deep reinforcement learning framework that leverages a Soft Actor-Critic algorithm with semi-Markov and Hawkes jump-diffusion models to simulate limit order book dynamics in non-Markovian financial markets.
key_findings bullet 1 · key_findings · validation V0
Incorporating adverse fill modeling and non-adverse fill probabilities boosts trading performance by enabling deep RL agents to navigate complex state-action spaces, minimize inventory risks, and outperform traditional stochastic control methods.
key_findings bullet 2 · key_findings · validation V0
Extensive simulations using real ES futures data verify the frameworks efficacy despite constant spread and midprice simplifications; findings highlight deep RL's promise yielding adaptive, realistic trading strategies to mitigate risk.
key_findings bullet 3 · key_findings · validation V0
This paper introduces an innovative approach by applying deep reinforcement learning to market-making, circumventing the typical Markov assumption. This novel method aligns with real-world complexities in electronic markets and limit order book research. Its originality and fresh perspective offer compelling insights into market microstructure dynamics, inspiring future research advancements significantly.
key_findings bullet 4 · key_findings · validation V0
Raw abstract and provenance
- … a deep reinforcement learning (RL) framework for an optimal market-making (MM) trading … In this way, the model can discover interesting trading strategies that are not …
Source row: 563 · abstract type: snippet