Bayesian Framework

The Bayesian Theorem

Bayesian Theorem

The Bayesian theorem states the following equation:$\text{pr}(A|B)=\frac{\text{pr}(B|A)\text{pr}(A)}{\text{pr}(B)}$where

• $\text{pr}(A\mid B)$ is a conditional probability, also called posterior.
• $\text{pr}(B\mid A)$ is also a conditional probability, also called the likelihood.
• $\text{pr}(A)$ and $\text{pr}(B)$ are probabilities of observing $A$ and $B$ respectively without any given conditions, are known as the prior probability and marginal probability.

Bayesian Inference

Estimator

Suppose a fixed parameter $\theta$ needs to be estimated. An estimator of $\theta$, written as $\hat{\theta }$ is a function that maps the observation space to the parameter space, which is naturally a random variable.

Bias

The bias of an estimator $\hat{\theta }$ is defined as $b(\theta )={\mathbb{E}}_{x|\theta }[\hat{\theta }]-\theta$ If $b(\theta )=0$, then we call the estimator unbiased, otherwise it is biased.