What is a conjugate family?

In Bayesian probability theory, if the posterior distribution p(θ | x) is in the same probability distribution family as the prior probability distribution p(θ), the prior and posterior are then called conjugate distributions, and the prior is called a conjugate prior for the likelihood function p(x | θ).

Which is the family of exponential distribution?

The normal, exponential, log-normal, gamma, chi-squared, beta, Dirichlet, Bernoulli, categorical, Poisson, geometric, inverse Gaussian, von Mises and von Mises-Fisher distributions are all exponential families. Some distributions are exponential families only if some of their parameters are held fixed.

What does conjugate mean in Bayesian?

In Bayesian probability theory, if the posterior distribution is in the same family of the prior distribution, then the prior and posterior are called conjugate distributions, and the prior is called the conjugate prior to the likelihood function.

What is conjugate prior of normal distribution?

A normal prior is conjugate to a normal likelihood with known σ. Data: x1,x2,…,xn. Normal likelihood. x1,x2,…,xn ∼ N(θ, σ2) Assume θ is our unknown parameter of interest, σ is known.

What is conjugate prior for exponential distribution?

For exponential families the likelihood is a simple standarized function of the parameter and we can define conjugate priors by mimicking the form of the likelihood. Multiplication of a likelihood and a prior that have the same exponential form yields a posterior that retains that form.

What is the conjugate prior of normal distribution?

If you have a conjugate prior this means that the prior comes from the same family of distributions and there is a closed-form solution for such problem, so the posterior distribution is directly available. This is exactly the case when you use normal prior for mean parameter of normal distribution.

Why conjugate prior distributions are useful in Bayesian statistics?

Why are conjugate priors useful? Since the posterior is from the same family of distributions as a conjugate prior, it is very easy evaluate the effects of the observed data on inference (practical). Conjugate priors can help defining priors in more complicated inference problems where conjugacy is not possible.

Why is conjugate prior useful?

Do exponential families have conjugate priors?

Exponential families have conjugate priors, an important property in Bayesian statistics. The posterior predictive distribution of an exponential-family random variable with a conjugate prior can always be written in closed form (provided that the normalizing factor of the exponential-family distribution can itself be written in closed form).

What is exponential family of distributions?

Exponential family. The exponential family of distributions provides a general framework for selecting a possible alternative parameterisation of the distribution, in terms of natural parameters, and for defining useful sample statistics, called the natural sufficient statistics of the family.

What is a conjugate prior in Bayesian statistics?

In Bayesian statistics a prior distribution is multiplied by a likelihood function and then normalised to produce a posterior distribution. In the case of a likelihood which belongs to an exponential family there exists a conjugate prior, which is often also in an exponential family.

What is a single-parameter exponential family?

A single-parameter exponential family is a set of probability distributions whose probability density function (or probability mass function, for the case of a discrete distribution) can be expressed in the form. where T(x), h(x), η(θ), and A(θ) are known functions.