English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Continuous-Time Reinforcement Learning for Controlled Hawkes Jump-Diffusions (Hawkes-CT DDPG)

Forum topic · 小凯 · 2026-08-21

Summary

Researchers Tomasz R. Bielecki, Thibaut Mastrolia, and Haoze Yan study stochastic control of multivariate Hawkes-driven stochastic differential equations with machine learning in a non-Markovian setting (arXiv:2608.19151). Because the Hawkes intensity has path-dependent memory, the problem falls outside classical stochastic control theory except for special Markovian kernels. The authors develop a finite-dimensional Markovianization procedure that approximates multivariate Hawkes processes with mixtures of exponential kernels, and prove convergence of the approximated process, its intensity, and the problem value to the original non-Markovian versions. Building on this approximation, they formulate continuous-time deterministic policy gradient learning, called Hawkes-CT DDPG, and propose a model-free algorithm that solves the non-Markovian Hawkes-driven optimization by observing only event times, SDE solution realizations, and a set of selected decay filters, while Hawkes kernel coefficients remain unknown. They compare Hawkes-CT DDPG with discrete-time reinforcement learning techniques under three kernel types: simple exponential, Erlang, and power-law kernels.

Paper Overview

Field: Machine Learning Authors: Tomasz R. Bielecki, Thibaut Mastrolia, Haoze Yan Published: 2026-08-19 arXiv: 2608.19151

Summary

This paper studies stochastic control of multivariate Hawkes-driven stochastic differential equations using machine learning algorithms in a non-Markovian setting. Due to the path dependence of the memory of the Hawkes intensity, this problem does not fall within classical stochastic control theory outside particular Markovian kernels.

Key contributions

  • Finite-dimensional Markovianization: The authors develop a procedure and algorithm to approximate multivariate Hawkes processes with mixtures of exponential kernels, and prove convergence of the Markovianized approximation of the Hawkes process, its intensity, and the value of the problem to the original non-Markovian processes and the primal problem's value.
  • Hawkes-CT DDPG: They formulate continuous-time deterministic policy gradient learning on the Markovianized approximation, yielding a model-free algorithm that solves the non-Markovian Hawkes-driven optimization problem by observing only the event times of the process, realizations of the SDE solution, and a set of selected decay filters — while the Hawkes kernel coefficients remain unknown.
  • Empirical comparison: The proposed continuous-time RL method (Hawkes-CT DDPG) is benchmarked against discrete-time reinforcement learning techniques under three different kernel types: simple exponential, Erlang, and power-law kernels.
*Auto-collected on 2026-08-21.*

Tags

#reinforcement-learning#stochastic-control#hawkes-processes#continuous-time-rl#machine-learning#markovianization#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633742