Paper Overview
Field: Machine Learning Authors: Tomasz R. Bielecki, Thibaut Mastrolia, Haoze Yan Published: 2026-08-19 arXiv: 2608.19151
Summary
This paper studies stochastic control of multivariate Hawkes-driven stochastic differential equations using machine learning algorithms in a non-Markovian setting. Due to the path dependence of the memory of the Hawkes intensity, this problem does not fall within classical stochastic control theory outside particular Markovian kernels.
Key contributions
- Finite-dimensional Markovianization: The authors develop a procedure and algorithm to approximate multivariate Hawkes processes with mixtures of exponential kernels, and prove convergence of the Markovianized approximation of the Hawkes process, its intensity, and the value of the problem to the original non-Markovian processes and the primal problem's value.
- Hawkes-CT DDPG: They formulate continuous-time deterministic policy gradient learning on the Markovianized approximation, yielding a model-free algorithm that solves the non-Markovian Hawkes-driven optimization problem by observing only the event times of the process, realizations of the SDE solution, and a set of selected decay filters — while the Hawkes kernel coefficients remain unknown.
- Empirical comparison: The proposed continuous-time RL method (Hawkes-CT DDPG) is benchmarked against discrete-time reinforcement learning techniques under three different kernel types: simple exponential, Erlang, and power-law kernels.