Skip to content

Add low-rank multivariate normal distribution #3288

@lingium

Description

@lingium

Description

If a low-rank structure exists (or is assumed to exist) in the covariance matrix of MVN, i.e., $$\Sigma=W W^\top + D$$, where $$W$$ is $$n\times m$$ ($$m<<n$$), $$D$$ is a positive definite diagonal matrix, then the determinant and the inverse can be reduced from $$O(n^3)$$ to $$O(nm^2+m^3)$$. A continuation of #3038.

For implementation, can refer to
https://en.wikipedia.org/wiki/Woodbury_matrix_identity
https://en.wikipedia.org/wiki/Matrix_determinant_lemma

numpyro
Low-rank Multivariate Normal distribution · Issue #387 · pyro-ppl/numpyro
https://github.com/pyro-ppl/numpyro/blob/master/numpyro/distributions/continuous.py

pytorch
Probability distributions - torch.distributions — PyTorch 2.10 documentation
pytorch/torch/distributions/lowrank_multivariate_normal.py at 14e348b7ad1b3472812f2b077020d80deaf6a787 · pytorch/pytorch

Applications:
Intermediate between mean-field and full-rank ADVI
AutoLowRankMultivariateNormal-numpyro

Expected Output

Maybe follow the multi_normal_XXX series?

real multi_normal_lowrank_lpdf(vector y | vector mu, matrix cov_factor, vector cov_diag)
also
real multi_normal_lowrank_lpdf(vectors y | vectors mu, matrix cov_factor, vector cov_diag)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions