Efficient EM-variational inference for nonparametric Hawkes process

The classic Hawkes process assumes the baseline intensity to be constant and the triggering kernel to be a parametric function. Differently, we present a generalization of the parametric Hawkes process by using a Bayesian nonparametric model called quadratic Gaussian Hawkes process. We model the baseline intensity and trigger kernel as the quadratic transformation of random trajectories drawn from a Gaussian process (GP) prior. We derive an analytical expression for the EM-variational inference algorithm by augmenting the latent branching structure of the Hawkes process to embed the variational Gaussian approximation into the EM framework naturally. We also use a series of schemes based on the sparse GP approximation to accelerate the inference algorithm. The results of synthetic and real data experiments show that the underlying baseline intensity and triggering kernel can be recovered efficiently and our model achieved superior performance in fitting capability and prediction accuracy compared to the state-of-the-art approaches.

1. https://data.cityofnewyork.us/Public-Safety/NYPD-Motor-Vehicle-Collisions/h9gi-nx95.

2. https://data.cityofnewyork.us/Transportation/2016-Green-Taxi-Trip-Data/hvrh-b6nb.

Authors and Affiliations

1. School of Computer Science and Engineering, University of New South Wales, Kensington, Australia

   Feng Zhou & Arcot Sowmya

2. School of Mathematics and Statistics, The University of Sydney, Sydney, Australia

   Simon Luo

3. Data Analytics for Research and Environment, Australian Research Council, Canberra, Australia

   Simon Luo

4. Data Science Institute, University of Technology Sydney, Ultimo, Australia

   Zhidong Li, Yang Wang & Fang Chen

5. School of Mathematics and Statistics, University of New South Wales, Kensington, Australia

   Xuhui Fan

Corresponding author

Correspondence to Zhidong Li.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Proof of lower-bound

The lower-bound \(Q(\mu (t),\phi (\tau )|\mu ^{(s)}(t),\phi ^{(s)}(\tau ))\) in Eq. (4) is induced as follows. Based on Jensen’s inequality, we have

where C is a constant because \(p_{ii}\) and \(p_{ij}\) are given in the E-step.

Analytical solution of ELBO

The \(\text {KL}\left( q(\mathbf {u}_f)||p(\mathbf {u}_f)\right) \) can be written as

where \(\text {Tr}(\cdot )\) means trace, \(|\cdot |\) means determinant and \(M_f\) is the dimensionality of \(\mathbf {u}_f\).

The last two terms in Eq. (8) have analytical solutions (Lloyd et al. 2015)

where \(\varPsi _f(z_f,z_f^\prime )=\int _0^Tk(z_f,t)k(t,z_f^\prime )dt\). For the squared exponential kernel, \(\varPsi _f\) can be written as (Lloyd et al. 2015)

where \(\text {erf}(\cdot )\) is Gauss error function and \(\bar{z}_f=(z_f+z_f^\prime )/2\).

The first term in Eq. (8) also has an analytical solution (Lloyd et al. 2015)

where \({\tilde{\sigma }}^2_f(t_i)\) is the diagonal entry of \({\tilde{\varSigma }}_f(t,t^\prime )\) in Eq. (7) at \(t_i\), C is the Euler-Mascheroni constant 0.57721566 and \(\tilde{G}(z)\) is a special case of the partial derivative of the confluent hyper-geometric function \(_1F_1(a,b,z)\) (Lloyd et al. 2015)

It is worth noting that \(\tilde{G}(z)\) does not need to be computed for inference. Actually we only need to know \(\tilde{G}(0)=0\) because \(\mathbf {m}_f^*=\mathbf {0}\) as we have shown in the section of inference speed up.

Matrix derivative of ELBO

Given \(\mathbf {m}_f=\mathbf {0}\), \(\text {ELBO}_\mu \) can be written as

If \(\mathbf {S}_f\) is symmetric, \(\partial \text {ELBO}_\mu /\partial \mathbf {S}_f\) can be written as

where \(\mathbf {I}\) denotes the identity matrix, \(\circ \) denotes the Hadamard (elementwise) product and \({\tilde{\sigma }}_f^2(t_i)=\theta _0^f-\mathbf {k}_{t_i z_f}\mathbf {K}_{z_f z_f}^{-1}\mathbf {k}_{z_f t_i}+\mathbf {k}_{t_i z_f}\mathbf {K}^{-1}_{z_f z_f}\mathbf {S}_f\mathbf {K}^{-1}_{z_f z_f}\mathbf {k}_{z_f t_i}\) is the diagonal entry of \({\tilde{\varSigma }}_f(t,t^\prime )\) in Eq. (7).

If \(\mathbf {S}_f\) is diagonal, \(\partial \text {ELBO}_\mu /\partial \mathbf {S}_f\) can be further simplified as

Similarly given \(\mathbf {m}_g=\mathbf {0}\), \(\text {ELBO}_\phi \) can be written as

If \(\mathbf {S}_g\) is symmetric, \(\partial \text {ELBO}_\phi /\partial \mathbf {S}_g\) can be written as

where \({\tilde{\sigma }}_g^2(\tau _{ij})=\theta _0^g-\mathbf {k}_{\tau _{ij} z_g}\mathbf {K}_{z_g z_g}^{-1}\mathbf {k}_{z_g \tau _{ij}}+\mathbf {k}_{\tau _{ij} z_g}\mathbf {K}^{-1}_{z_g z_g}\mathbf {S}_g\mathbf {K}^{-1}_{z_g z_g} \mathbf {k}_{z_g \tau _{ij}}\) is the diagonal entry of \({\tilde{\varSigma }}_g(\tau ,\tau ^\prime )\).

If \(\mathbf {S}_g\) is diagonal, \(\partial \text {ELBO}_\phi /\partial \mathbf {S}_g\) can be further simplified as

Reprints and permissions