# Chapter 3 Mathematical statistic Trick

## 3.1 Normal distribution as exponential family

Theorem: If the density function with the form: $f(x) \propto \exp \left\{-\frac{1}{2} x^{T} A x+B x\right\}$

Then, we have $$X\sim N\left(\left(B A^{-1}\right)^{T}, A^{-1}\right)$$

## 3.4 Vector to diagonal matrix:

(source)[https://mathoverflow.net/questions/55820/vector-to-diagonal-matrix] Let $$E_i$$ be the $$n\times n$$ matrix with a 1 on position (i,i) and zeros else where. Similarly let $$e_i$$ be a row vector with 1 on position (1,i) and zero else where, then: $D=\sum_{i=1}^{n} E_{i} v e_{i}$ D is the diagonal matrix with its entry is vector $$v$$.

## 3.5 Gaussian Integral Trick

From the blog Gaussian integral Trick , there are many aspects in this blog, here is the most useful for me part: $\int \exp \left(-a y^{2}+x y\right) d y=\sqrt{\frac{\pi}{a}} \exp \left(\frac{1}{4 a} x^{2}\right)$

Using this technique, we can prove the “Objective Bayesian Methods for Model Selection: Introduction and Comparison” example under difficulty 3 that vague proper prior usually gives bad answers. Vague parameters usually have strongly influence on the Bayes factor.

Vector form:

“Completion of squares” trick: $\frac{1}{2} z^{T} A z+b^{T} z+c=\frac{1}{2}\left(z+A^{-1} b\right)^{T} A\left(z+A^{-1} b\right)+c-\frac{1}{2} b^{T} A^{-1} b$

Then use the fact from multivariate Gaussian:

$\frac{1}{(2 \pi)^{n / 2}|\Sigma|^{1 / 2}} \int_{\mathbf{R}^{n}} \exp \left(-\frac{1}{2}(x-\mu)^{T} \Sigma^{-1}(x-\mu)\right)=1$ or equivalently $\int_{\mathbf{R}^{n}} \exp \left(-\frac{1}{2}(x-\mu)^{T} \Sigma^{-1}(x-\mu)\right)=(2 \pi)^{n / 2}|\Sigma|^{1 / 2}$

So we have: $\int_{x \in \mathbf{R}^{n}} \exp \left(-\frac{1}{2} x^{T} A x-x^{T} b-c\right) d x=\frac{(2 \pi)^{n / 2}}{|A|^{1 / 2} \exp \left(c-b^{T} A^{-1} b\right)}$

## 3.6 Asymptotic issues

The case where $$\log \pi(\boldsymbol{\theta})=O_{p}(1)$$, then $$n^{-1} \log \pi(\boldsymbol{\theta}) \rightarrow 0$$ as $$n\rightarrow \infty$$.

Las of large numbers: 为啥是这个形式 $n^{-1} \log \left\{f\left(\boldsymbol{X}_{n} | \boldsymbol{\theta}\right) \pi(\boldsymbol{\theta})\right\} \rightarrow \int \log \left\{f(\boldsymbol{x} | \boldsymbol{\theta}) \pi_{0}(\boldsymbol{\theta})\right\} d G(\boldsymbol{x})$ 这个形式和大数定律联系在哪里。

## 3.7 Matrix inverse

$(A+D)^{-1}=D^{-1}\left(I+A D^{-1}\right)^{-1}$

If $$n=k$$ and $$U=V=I_{n}$$ is the identity matrix, then \begin{aligned} (A+B)^{-1} &=A^{-1}-A^{-1}\left(B^{-1}+A^{-1}\right)^{-1} A^{-1} \\ &=A^{-1}-A^{-1}\left(A B^{-1}+I\right)^{-1} \end{aligned} Continuing with the merging of the terms of the far right-hand side of the above equation results in Hua’s identity $(A+B)^{-1}=A^{-1}-\left(A+A B^{-1} A\right)^{-1}$ Another useful form of the same identity is $(A-B)^{-1}=A^{-1}+A^{-1} B(A-B)^{-1}$ which has a recursive structure that yields $(A-B)^{-1}=\sum_{k=0}^{\infty}\left(A^{-1} B\right)^{k} A^{-1}$ This form can be used in perturbative expansions where $$B$$ is a perturbation of $$A$$.

Woodbury matrix identity $(A+U C V)^{-1}=A^{-1}-A^{-1} U\left(C^{-1}+V A^{-1} U\right)^{-1} V A^{-1}$