von Mises–Fisher distribution

In directional statistics, the von Mises–Fisher distribution (named after Richard von Mises and Ronald Fisher), is a probability distribution on the $(p-1)$-sphere in $\mathbb {R} ^{p}$. If $p=2$ the distribution reduces to the von Mises distribution on the circle.

Definition

The probability density function of the von Mises–Fisher distribution for the random p-dimensional unit vector $\mathbf {x}$ is given by:

$f_{p}(\mathbf {x} ;{\boldsymbol {\mu }},\kappa )=C_{p}(\kappa )\exp \left({\kappa {\boldsymbol {\mu }}^{\mathsf {T}}\mathbf {x} }\right),$

where $\kappa \geq 0,\left\Vert {\boldsymbol {\mu }}\right\Vert =1$ and the normalization constant $C_{p}(\kappa )$ is equal to

$C_{p}(\kappa )={\frac {\kappa ^{p/2-1}}{(2\pi )^{p/2}I_{p/2-1}(\kappa )}},$

where $I_{v}$ denotes the modified Bessel function of the first kind at order $v$. If $p=3$, the normalization constant reduces to

$C_{3}(\kappa )={\frac {\kappa }{4\pi \sinh \kappa }}={\frac {\kappa }{2\pi (e^{\kappa }-e^{-\kappa })}}.$

The parameters ${\boldsymbol {\mu }}$ and $\kappa$ are called the mean direction and concentration parameter, respectively. The greater the value of $\kappa$, the higher the concentration of the distribution around the mean direction ${\boldsymbol {\mu }}$. The distribution is unimodal for $\kappa >0$, and is uniform on the sphere for $\kappa =0$.

The von Mises–Fisher distribution for $p=3$ is also called the Fisher distribution. It was first used to model the interaction of electric dipoles in an electric field. Other applications are found in geology, bioinformatics, and text mining.

Note on the normalization constant

In the textbook by Mardia and Jupp, the normalization constant given for the Von Mises Fisher probability density is apparently different from the one given here: $C_{p}(\kappa )$. In that book, the normalization constant is specified as:

$C_{p}^{*}(\kappa )={\frac {({\frac {\kappa }{2}})^{p/2-1}}{\Gamma (p/2)I_{p/2-1}(\kappa )}}$

This is resolved by noting that Mardia and Jupp give the density "with respect to the uniform distribution", while the density here is specified in the usual way, with respect to Lebesgue measure. The density (w.r.t. Lebesgue measure) of the uniform distribution is the reciprocal of the surface area of the (p-1)-sphere, so that the uniform density function is given by the constant:

$C_{p}(0)={\frac {\Gamma (p/2)}{2\pi ^{p/2}}}$

It then follows that:

$C_{p}^{*}(\kappa )={\frac {C_{p}(\kappa )}{C_{p}(0)}}$

While the value for $C_{p}(0)$ was derived above via the surface area, the same result may be obtained by setting $\kappa =0$ in the above formula for $C_{p}(\kappa )$. This can be done by noting that the series expansion for $I_{p/2-1}(\kappa )$ divided by $\kappa ^{p/2-1}$ has but one non-zero term at $\kappa =0$. (To evaluate that term, one needs to use the definition $0^{0}=1$.)

Relation to normal distribution

Starting from a normal distribution with isotropic covariance $\kappa ^{-1}\mathbf {I}$ and mean ${\boldsymbol {\mu }}$ of length $r>0$, whose density function is:

$G_{p}(\mathbf {x} ;{\boldsymbol {\mu }},\kappa )=\left({\sqrt {\frac {\kappa }{2\pi }}}\right)^{p}\exp \left(-\kappa {\frac {(\mathbf {x} -{\boldsymbol {\mu }})'(\mathbf {x} -{\boldsymbol {\mu }})}{2}}\right),$

the Von Mises–Fisher distribution is obtained by conditioning on $\left\|\mathbf {x} \right\|=1$. By expanding

$(\mathbf {x} -{\boldsymbol {\mu }})'(\mathbf {x} -{\boldsymbol {\mu }})=\mathbf {x} '\mathbf {x} +{\boldsymbol {\mu }}'{\boldsymbol {\mu }}-2{\boldsymbol {\mu }}'\mathbf {x} ,$

and using the fact that the first two right-hand-side terms are fixed, the Von Mises-Fisher density, $f_{p}(\mathbf {x} ;r^{-1}{\boldsymbol {\mu }},r\kappa )$ is recovered by recomputing the normalization constant by integrating $\mathbf {x}$ over the unit sphere. If $r=0$, we get the uniform distribution, with density $f_{p}(\mathbf {x} ;{\boldsymbol {0}},0)$.

More succinctly, the restriction of any isotropic multivariate normal density to the unit hypersphere, gives a Von Mises-Fisher density, up to normalization.

This construction can be generalized by starting with a normal distribution with a general covariance matrix, in which case conditioning on $\left\|\mathbf {x} \right\|=1$ gives the Fisher-Bingham distribution.

Estimation of parameters

Mean direction

A series of N independent unit vectors $x_{i}$ are drawn from a von Mises–Fisher distribution. The maximum likelihood estimates of the mean direction $\mu$ is simply the normalized arithmetic mean, a sufficient statistic:

$\mu ={\bar {x}}/{\bar {R}},{\text{where }}{\bar {x}}={\frac {1}{N}}\sum _{i}^{N}x_{i},{\text{and }}{\bar {R}}=\|{\bar {x}}\|,$

Concentration parameter

Use the Bessel function of the first kind to define

$A_{p}(\kappa )={\frac {I_{p/2}(\kappa )}{I_{p/2-1}(\kappa )}}.$

Then:

$\kappa =A_{p}^{-1}({\bar {R}}).$

Thus $\kappa$ is the solution to

$A_{p}(\kappa )={\frac {\left\|\sum _{i}^{N}x_{i}\right\|}{N}}={\bar {R}}.$

A simple approximation to $\kappa$ is (Sra, 2011)

${\hat {\kappa }}={\frac {{\bar {R}}(p-{\bar {R}}^{2})}{1-{\bar {R}}^{2}}},$

A more accurate inversion can be obtained by iterating the Newton method a few times

${\hat {\kappa }}_{1}={\hat {\kappa }}-{\frac {A_{p}({\hat {\kappa }})-{\bar {R}}}{1-A_{p}({\hat {\kappa }})^{2}-{\frac {p-1}{\hat {\kappa }}}A_{p}({\hat {\kappa }})}},$

${\hat {\kappa }}_{2}={\hat {\kappa }}_{1}-{\frac {A_{p}({\hat {\kappa }}_{1})-{\bar {R}}}{1-A_{p}({\hat {\kappa }}_{1})^{2}-{\frac {p-1}{{\hat {\kappa }}_{1}}}A_{p}({\hat {\kappa }}_{1})}}.$

Standard error

For N ≥ 25, the estimated spherical standard error of the sample mean direction can be computed as:

${\hat {\sigma }}=\left({\frac {d}{N{\bar {R}}^{2}}}\right)^{1/2}$

where

$d=1-{\frac {1}{N}}\sum _{i}^{N}\left(\mu ^{T}x_{i}\right)^{2}$

It is then possible to approximate a $100(1-\alpha )\%$ a spherical confidence interval (a confidence cone) about $\mu$ with semi-vertical angle:

$q=\arcsin \left(e_{\alpha }^{1/2}{\hat {\sigma }}\right),$ where $e_{\alpha }=-\ln(\alpha ).$

For example, for a 95% confidence cone, $\alpha =0.05,e_{\alpha }=-\ln(0.05)=2.996,$ and thus $q=\arcsin(1.731{\hat {\sigma }}).$

Expected value

The expected value of the Von Mises–Fisher distribution is not on the unit hypersphere, but instead has a length of less than one. This length is given by $A_{p}(\kappa )$ as defined above. For a Von Mises–Fisher distribution with mean direction ${\boldsymbol {\mu }}$ and concentration $\kappa >0$, the expected value is:

$A_{p}(\kappa ){\boldsymbol {\mu }}$.

For $\kappa =0$, the expected value is at the origin. For finite $\kappa >0$, the length of the expected value, is strictly between zero and one and is a monotonic rising function of $\kappa$.

The empirical mean (arithmetic average) of a collection of points on the unit hypersphere behaves in a similar manner, being close to the origin for widely spread data and close to the sphere for concentrated data. Indeed, for the Von Mises–Fisher distribution, the expected value of the maximum-likelihood estimate based on a collection of points is equal to the empirical mean of those points.

Entropy and KL divergence

The expected value can be used to compute differential entropy and KL divergence.

The differential entropy of $f_{p}(\mathbf {x} ;{\boldsymbol {\mu }},\kappa )$ is:

$-\log f_{p}(A_{p}(\kappa ){\boldsymbol {\mu }};{\boldsymbol {\mu }},\kappa )=-\log C_{p}(\kappa )-\kappa A_{p}(\kappa )$.

Notice that the entropy is a function of $\kappa$ only.

The KL divergence between $f_{p}(\mathbf {x} ;{\boldsymbol {\mu _{0}}},\kappa _{0})$ and $f_{p}(\mathbf {x} ;{\boldsymbol {\mu _{1}}},\kappa _{1})$ is:

$\log {\frac {f_{p}(A_{p}(\kappa _{0}){\boldsymbol {\mu _{0}}};{\boldsymbol {\mu _{0}}},\kappa _{0})}{f_{p}(A_{p}(\kappa _{0}){\boldsymbol {\mu _{0}}};{\boldsymbol {\mu _{1}}},\kappa _{1})}}$

Transformation

Von Mises-Fisher (VMF) distributions are closed under orthogonal linear transforms. Let $\mathbf {U}$ be a $p$-by-$p$ orthogonal matrix. Let $\mathbf {x} \sim {\text{VMF}}({\boldsymbol {\mu }},\kappa )$ and apply the invertible linear transform: $\mathbf {y} =\mathbf {Ux}$. The inverse transform is $\mathbf {x} =\mathbf {U'y}$, because the inverse of an orthogonal matrix is its transpose: $\mathbf {U} ^{-1}=\mathbf {U} '$. The Jacobian of the transform is $\mathbf {U}$, for which the absolute value of its determinant is 1, also because of the orthogonality. Using these facts and the form of the VMF density, it follows that:

$\mathbf {y} \sim {\text{VMF}}(\mathbf {U} {\boldsymbol {\mu }},\kappa ).$

One may verify that since ${\boldsymbol {\mu }}$ and $\mathbf {x}$ are unit vectors, then by the orthogonality, so are $\mathbf {U} {\boldsymbol {\mu }}$ and $\mathbf {y}$.

Generalizations

Further information: Random matrix

The matrix von Mises-Fisher distribution (also known as matrix Langevin distribution) has the density

$f_{n,p}(\mathbf {X} ;\mathbf {F} )\propto \exp(\operatorname {tr} (\mathbf {F} ^{\mathsf {T}}\mathbf {X} ))$

supported on the Stiefel manifold of $n\times p$ orthonormal p-frames $\mathbf {X}$, where $\mathbf {F}$ is an arbitrary $n\times p$ real matrix.

Distribution of polar angle

For $p=3$, the angle θ between $\mathbf {x}$ and ${\boldsymbol {\mu }}$ satisfies $\cos \theta ={\boldsymbol {\mu }}^{\mathsf {T}}\mathbf {x}$. It has the distribution

$p(\theta )=\int d^{2}xf(x;{\boldsymbol {\mu }},\kappa )\,\delta \left(\theta -{\text{arc cos}}({\boldsymbol {\mu }}^{\mathsf {T}}\mathbf {x} )\right)$,

which can be easily evaluated as

$p(\theta )=2\pi C_{3}(\kappa )\,\sin \theta \,e^{\kappa \cos \theta }$.

See also

• Kent distribution, a related distribution on the two-dimensional unit sphere
• von Mises distribution, von Mises–Fisher distribution where p = 2, the one-dimensional unit circle
• Bivariate von Mises distribution
• Directional statistics

Further reading

• Dhillon, I., Sra, S. (2003) "Modeling Data using Directional Distributions". Tech. rep., University of Texas, Austin.
• Banerjee, A., Dhillon, I. S., Ghosh, J., & Sra, S. (2005). "Clustering on the unit hypersphere using von Mises-Fisher distributions". Journal of Machine Learning Research, 6(Sep), 1345-1382.
• Sra, S. (2011). "A short note on parameter approximation for von Mises-Fisher distributions: And a fast implementation of I_s(x)". Computational Statistics. 27: 177–190. CiteSeerX 10.1.1.186.1887. doi:10.1007/s00180-011-0232-x. S2CID 3654195.