# 9. Probability distributions¶

PyMC provides a large suite of built-in probability distributions. For each distribution, it provides:

• A function that evaluates its log-probability or log-density: normal_like().
• A function that draws random variables: rnormal().
• A function that computes the expectation associated with the distribution: normal_expval().
• A Stochastic subclass generated from the distribution: Normal.

This section describes the likelihood functions of these distributions.

## 9.1. Discrete distributions¶

pymc.distributions.bernoulli_like(x, p)[source]

Bernoulli log-likelihood

The Bernoulli distribution describes the probability of successes (x=1) and failures (x=0).

$f(x \mid p) = p^{x} (1-p)^{1-x}$
Parameters: x : Series of successes (1) and failures (0). $$x=0,1$$ p : Probability of success. $$0 < p < 1$$. >>> from pymc import bernoulli_like >>> bernoulli_like([0,1,0,1], .4) -2.854232711280291 

Note

• $$E(x)= p$$
• $$Var(x)= p(1-p)$$
pymc.distributions.binomial_like(x, n, p)[source]

Binomial log-likelihood. The discrete probability distribution of the number of successes in a sequence of n independent yes/no experiments, each of which yields success with probability p.

$f(x \mid n, p) = \frac{n!}{x!(n-x)!} p^x (1-p)^{n-x}$
Parameters: x : [int] Number of successes, > 0. n : [int] Number of Bernoulli trials, > x. p : Probability of success in each trial, $$p \in [0,1]$$.

Note

• $$E(X)=np$$
• $$Var(X)=np(1-p)$$
pymc.distributions.categorical_like(x, p)[source]

Categorical log-likelihood. The most general discrete distribution.

$f(x=i \mid p) = p_i$

for $$i \in 0 \ldots k-1$$.

Parameters: x : [int] $$x \in 0\ldots k-1$$ p : [float] $$p > 0$$, $$\sum p = 1$$
pymc.distributions.discrete_uniform_like(x, lower, upper)[source]

Discrete uniform log-likelihood.

$f(x \mid lower, upper) = \frac{1}{upper-lower+1}$
Parameters: x : [int] $$x \in \{lower, lower+1, \ldots, upper-1, upper\}$$ lower : Lower limit. upper : Upper limit (upper > lower).
pymc.distributions.geometric_like(x, p)[source]

Geometric log-likelihood. The probability that the first success in a sequence of Bernoulli trials occurs on the x’th trial.

$f(x \mid p) = p(1-p)^{x-1}$
Parameters: x : [int] Number of trials before first success (x > 0). p : Probability of success on an individual trial, $$p \in [0,1]$$

Note

• $$E(X)=1/p$$
• $$Var(X)=\frac{1-p}{p^2}$$
pymc.distributions.hypergeometric_like(x, n, m, N)[source]

Hypergeometric log-likelihood.

Discrete probability distribution that describes the number of successes in a sequence of draws from a finite population without replacement.

$\begin{split}f(x \mid n, m, N) = \frac{\left({ \begin{array}{c} {m} \\ {x} \\ \end{array} }\right)\left({ \begin{array}{c} {N-m} \\ {n-x} \\ \end{array}}\right)}{\left({ \begin{array}{c} {N} \\ {n} \\ \end{array}}\right)}\end{split}$
Parameters: x : [int] Number of successes in a sample drawn from a population. n : [int] Size of sample drawn from the population. m : [int] Number of successes in the population. N : [int] Total number of units in the population.

Note

$$E(X) = \frac{n n}{N}$$

pymc.distributions.negative_binomial_like(x, mu, alpha)[source]

Negative binomial log-likelihood.

The negative binomial distribution describes a Poisson random variable whose rate parameter is gamma distributed. PyMC’s chosen parameterization is based on this mixture interpretation.

$f(x \mid \mu, \alpha) = \frac{\Gamma(x+\alpha)}{x! \Gamma(\alpha)} (\alpha/(\mu+\alpha))^\alpha (\mu/(\mu+\alpha))^x$
Parameters: x : x = 0,1,2,… mu : mu > 0 alpha : alpha > 0

Note

• $$E[x]=\mu$$
• In Wikipedia’s parameterization, $$r=\alpha$$ $$p=\alpha/(\mu+\alpha)$$ $$\mu=rp/(1-p)$$
pymc.distributions.poisson_like(x, mu)[source]

Poisson log-likelihood.

The Poisson is a discrete probability distribution. It is often used to model the number of events occurring in a fixed period of time when the times at which events occur are independent. The Poisson distribution can be derived as a limiting case of the binomial distribution.

$f(x \mid \mu) = \frac{e^{-\mu}\mu^x}{x!}$
Parameters: x : [int] $$x \in {0,1,2,...}$$ mu : Expected number of occurrences during the given interval, $$\mu \geq 0$$.

Note

• $$E(x)=\mu$$
• $$Var(x)=\mu$$
pymc.distributions.truncated_poisson_like(x, mu, k)[source]

Truncated Poisson log-likelihood.

The Truncated Poisson is a discrete probability distribution that is arbitrarily truncated to be greater than some minimum value k. For example, zero-truncated Poisson distributions can be used to model counts that are constrained to be non-negative.

$f(x \mid \mu, k) = \frac{e^{-\mu}\mu^x}{x!(1-F(k|\mu))}$
Parameters: x : [int] $$x \in {0,1,2,...}$$ mu : Expected number of occurrences during the given interval, $$\mu \geq 0$$. k : Truncation point representing the minimum allowable value.

Note

• $$E(x)=\frac{\mu}{1-F(k|\mu)}$$
• $$Var(x)=\frac{\mu}{1-F(k|\mu)}$$

## 9.2. Continuous distributions¶

pymc.distributions.beta_like(x, alpha, beta)[source]

Beta log-likelihood. The conjugate prior for the parameter $$p$$ of the binomial distribution.

$f(x \mid \alpha, \beta) = \frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha) \Gamma(\beta)} x^{\alpha - 1} (1 - x)^{\beta - 1}$
Parameters: x : 0 < x < 1 alpha : alpha > 0 beta : beta > 0 >>> from pymc import beta_like >>> beta_like(.4,1,2) 0.182321556793954 

Note

• $$E(X)=\frac{\alpha}{\alpha+\beta}$$
• $$Var(X)=\frac{\alpha \beta}{(\alpha+\beta)^2(\alpha+\beta+1)}$$
pymc.distributions.cauchy_like(x, alpha, beta)[source]

Cauchy log-likelihood. The Cauchy distribution is also known as the Lorentz or the Breit-Wigner distribution.

$f(x \mid \alpha, \beta) = \frac{1}{\pi \beta [1 + (\frac{x-\alpha}{\beta})^2]}$
Parameters: alpha : Location parameter. beta : Scale parameter > 0.

Note

• Mode and median are at alpha.
pymc.distributions.chi2_like(x, nu)[source]

Chi-squared $$\chi^2$$ log-likelihood.

$f(x \mid \nu) = \frac{x^{(\nu-2)/2}e^{-x/2}}{2^{\nu/2}\Gamma(\nu/2)}$
Parameters: x : > 0 nu : [int] Degrees of freedom ( nu > 0 )

Note

• $$E(X)=\nu$$
• $$Var(X)=2\nu$$
pymc.distributions.degenerate_like(x, k)[source]

Degenerate log-likelihood.

$\begin{split}f(x \mid k) = \left\{ \begin{matrix} 1 \text{ if } x = k \\ 0 \text{ if } x \ne k\end{matrix} \right.\end{split}$
Parameters: x : Input value. k : Degenerate value.
pymc.distributions.exponential_like(x, beta)[source]

Exponential log-likelihood.

The exponential distribution is a special case of the gamma distribution with alpha=1. It often describes the time until an event.

$f(x \mid \beta) = \beta e^{-\beta x}$
Parameters: x : x > 0 beta : Rate parameter (beta > 0).

Note

• $$E(X) = 1/\beta$$
• $$Var(X) = 1/\beta^2$$
• PyMC’s beta is named ‘lambda’ by Wikipedia, SciPy, Wolfram MathWorld and other sources.
pymc.distributions.exponweib_like(x, alpha, k, loc=0, scale=1)[source]

Exponentiated Weibull log-likelihood.

The exponentiated Weibull distribution is a generalization of the Weibull family. Its value lies in being able to model monotone and non-monotone failure rates.

$\begin{split}f(x \mid \alpha,k,loc,scale) & = \frac{\alpha k}{scale} (1-e^{-z^k})^{\alpha-1} e^{-z^k} z^{k-1} \\ z & = \frac{x-loc}{scale}\end{split}$
Parameters: x : x > 0 alpha : Shape parameter k : k > 0 loc : Location parameter scale : Scale parameter (scale > 0).
pymc.distributions.gamma_like(x, alpha, beta)[source]

Gamma log-likelihood.

Represents the sum of alpha exponentially distributed random variables, each of which has rate parameter beta.

$f(x \mid \alpha, \beta) = \frac{\beta^{\alpha}x^{\alpha-1}e^{-\beta x}}{\Gamma(\alpha)}$
Parameters: x : math:x ge 0 alpha : Shape parameter (alpha > 0). beta : Rate parameter (beta > 0).

Note

• $$E(X) = \frac{\alpha}{\beta}$$
• $$Var(X) = \frac{\alpha}{\beta^2}$$
pymc.distributions.half_cauchy_like(x, alpha, beta)[source]

Half-Cauchy log-likelihood. Simply the absolute value of Cauchy.

$f(x \mid \alpha, \beta) = \frac{2}{\pi \beta [1 + (\frac{x-\alpha}{\beta})^2]}$
Parameters: alpha : Location parameter. beta : Scale parameter (beta > 0).

Note

• x must be non-negative.
pymc.distributions.half_normal_like(x, tau)[source]

Half-normal log-likelihood, a normal distribution with mean 0 limited to the domain $$x \in [0, \infty)$$.

$f(x \mid \tau) = \sqrt{\frac{2\tau}{\pi}}\exp\left\{ {\frac{-x^2 \tau}{2}}\right\}$
Parameters: x : $$x \ge 0$$ tau : tau > 0
pymc.distributions.inverse_gamma_like(x, alpha, beta)[source]

Inverse gamma log-likelihood, the reciprocal of the gamma distribution.

$f(x \mid \alpha, \beta) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} x^{-\alpha - 1} \exp\left(\frac{-\beta}{x}\right)$
Parameters: x : x > 0 alpha : Shape parameter (alpha > 0). beta : Scale parameter (beta > 0).

Note

$$E(X)=\frac{\beta}{\alpha-1}$$ for $$\alpha > 1$$ $$Var(X)=\frac{\beta^2}{(\alpha-1)^2(\alpha-2)}$$ for $$\alpha > 2$$

pymc.distributions.laplace_like(x, mu, tau)[source]

Laplace (double exponential) log-likelihood.

The Laplace (or double exponential) distribution describes the difference between two independent, identically distributed exponential events. It is often used as a heavier-tailed alternative to the normal.

$f(x \mid \mu, \tau) = \frac{\tau}{2}e^{-\tau |x-\mu|}$
Parameters: x : $$-\infty < x < \infty$$ mu : Location parameter $$-\infty < mu < \infty$$ tau : Scale parameter $$\tau > 0$$

Note

• $$E(X) = \mu$$
• $$Var(X) = \frac{2}{\tau^2}$$
pymc.distributions.logistic_like(x, mu, tau)[source]

Logistic log-likelihood.

The logistic distribution is often used as a growth model; for example, populations, markets. Resembles a heavy-tailed normal distribution.

$f(x \mid \mu, tau) = \frac{\tau \exp(-\tau[x-\mu])}{[1 + \exp(-\tau[x-\mu])]^2}$
Parameters: x : $$-\infty < x < \infty$$ mu : Location parameter $$-\infty < mu < \infty$$ tau : Scale parameter (tau > 0)

Note

• $$E(X) = \mu$$
• $$Var(X) = \frac{\pi^2}{3\tau^2}$$
pymc.distributions.lognormal_like(x, mu, tau)[source]

Log-normal log-likelihood.

Distribution of any random variable whose logarithm is normally distributed. A variable might be modeled as log-normal if it can be thought of as the multiplicative product of many small independent factors.

$f(x \mid \mu, \tau) = \sqrt{\frac{\tau}{2\pi}}\frac{ \exp\left\{ -\frac{\tau}{2} (\ln(x)-\mu)^2 \right\}}{x}$
Parameters: x : x > 0 mu : Location parameter. tau : Scale parameter (tau > 0).

Note

$$E(X)=e^{\mu+\frac{1}{2\tau}}$$ $$Var(X)=(e^{1/\tau}-1)e^{2\mu+\frac{1}{\tau}}$$

pymc.distributions.noncentral_t_like(x, mu, lam, nu)[source]

Non-central Student’s T log-likelihood.

Describes a normal variable whose precision is gamma distributed.

$f(x|\mu,\lambda,\nu) = \frac{\Gamma(\frac{\nu + 1}{2})}{\Gamma(\frac{\nu}{2})} \left(\frac{\lambda}{\pi\nu}\right)^{\frac{1}{2}} \left[1+\frac{\lambda(x-\mu)^2}{\nu}\right]^{-\frac{\nu+1}{2}}$
Parameters: x : Input data. mu : Location parameter. lambda : Scale parameter. nu : Degrees of freedom.
pymc.distributions.normal_like(x, mu, tau)[source]

Normal log-likelihood.

$f(x \mid \mu, \tau) = \sqrt{\frac{\tau}{2\pi}} \exp\left\{ -\frac{\tau}{2} (x-\mu)^2 \right\}$
Parameters: x : Input data. mu : Mean of the distribution. tau : Precision of the distribution, which corresponds to $$1/\sigma^2$$ (tau > 0).

Note

• $$E(X) = \mu$$
• $$Var(X) = 1/\tau$$
pymc.distributions.pareto_like(x, alpha, m)[source]

Pareto log-likelihood. The Pareto is a continuous, positive probability distribution with two parameters. It is often used to characterize wealth distribution, or other examples of the 80/20 rule.

$f(x \mid \alpha, m) = \frac{\alpha m^{\alpha}}{x^{\alpha+1}}$
Parameters: x : Input data (x > m) alpha : Shape parameter (alpha>0) m : Scale parameter (m>0)

Note

• $$E(x)=\frac{\alpha m}{\alpha-1} if \alpha > 1$$
• $$Var(x)=\frac{m^2 \alpha}{(\alpha-1)^2(\alpha-2)} if \alpha > 2$$
pymc.distributions.skew_normal_like(x, mu, tau, alpha)[source]

Azzalini’s skew-normal log-likelihood

$f(x \mid \mu, \tau, \alpha) = 2 \Phi((x-\mu)\sqrt{\tau}\alpha) \phi(x,\mu,\tau)$

where :math:Phi is the normal CDF and :math: phi is the normal PDF.

Parameters: x : Input data. mu : Mean of the distribution. tau : Precision of the distribution (> 0). alpha : Shape parameter of the distribution.
pymc.distributions.t_like(x, nu)[source]

Student’s T log-likelihood.

Describes a zero-mean normal variable whose precision is gamma distributed. Alternatively, describes the mean of several zero-mean normal random variables divided by their sample standard deviation.

$f(x \mid \nu) = \frac{\Gamma(\frac{\nu+1}{2})}{\Gamma(\frac{\nu}{2}) \sqrt{\nu\pi}} \left( 1 + \frac{x^2}{\nu} \right)^{-\frac{\nu+1}{2}}$
Parameters: x : Input data. nu : Degrees of freedom.
pymc.distributions.truncated_normal_like(x, mu, tau, a=None, b=None)[source]

Truncated normal log-likelihood.

$f(x \mid \mu, \tau, a, b) = \frac{\phi(\frac{x-\mu}{\sigma})} {\Phi(\frac{b-\mu}{\sigma}) - \Phi(\frac{a-\mu}{\sigma})},$

where $$\sigma^2=1/\tau$$, phi is the standard normal PDF and Phi is the standard normal CDF.

Parameters: x : Input data. mu : Mean of the distribution. tau : Precision of the distribution, which corresponds to 1/sigma**2 (tau > 0). a : Left bound of the distribution. b : Right bound of the distribution.
pymc.distributions.truncated_pareto_like(x, alpha, m, b)[source]

Truncated Pareto log-likelihood. The Pareto is a continuous, positive probability distribution with two parameters. It is often used to characterize wealth distribution, or other examples of the 80/20 rule.

$f(x \mid \alpha, m, b) = \frac{\alpha m^{\alpha} x^{-\alpha}}{1-(m/b)**{\alpha}}$
Parameters: x : Input data (x > m) alpha : Shape parameter (alpha>0) m : Scale parameter (m>0) b : Upper bound (b>m)
pymc.distributions.uniform_like(x, lower, upper)[source]

Uniform log-likelihood.

$f(x \mid lower, upper) = \frac{1}{upper-lower}$
Parameters: x : $$lower \leq x \leq upper$$ lower : Lower limit. upper : Upper limit (upper > lower).
pymc.distributions.von_mises_like(x, mu, kappa)[source]

von Mises log-likelihood.

$f(x \mid \mu, k) = \frac{e^{k \cos(x - \mu)}}{2 \pi I_0(k)}$

where I_0 is the modified Bessel function of order 0.

Parameters: x : Input data. mu : Mean of the distribution. kappa : Dispersion of the distribution

Note

• $$E(X) = \mu$$
pymc.distributions.weibull_like(x, alpha, beta)[source]

Weibull log-likelihood

$f(x \mid \alpha, \beta) = \frac{\alpha x^{\alpha - 1} \exp(-(\frac{x}{\beta})^{\alpha})}{\beta^\alpha}$
Parameters: x : $$x \ge 0$$ alpha : alpha > 0 beta : beta > 0

Note

• $$E(x)=\beta \Gamma(1+\frac{1}{\alpha})$$
• $$Var(x)=\beta^2 \Gamma(1+\frac{2}{\alpha} - \mu^2)$$

## 9.3. Multivariate discrete distributions¶

pymc.distributions.multivariate_hypergeometric_like(x, m)[source]

Multivariate hypergeometric log-likelihood

Describes the probability of drawing x[i] elements of the ith category, when the number of items in each category is given by m.

$\begin{split}\frac{\prod_i \left({ \begin{array}{c} {m_i} \\ {x_i} \\ \end{array}}\right)}{\left({ \begin{array}{c} {N} \\ {n} \\ \end{array}}\right)}\end{split}$

where $$N = \sum_i m_i$$ and $$n = \sum_i x_i$$.

Parameters: x : [int sequence] Number of draws from each category, (x < m). m : [int sequence] Number of items in each categoy.
pymc.distributions.multinomial_like(x, n, p)[source]

Multinomial log-likelihood.

Generalization of the binomial distribution, but instead of each trial resulting in “success” or “failure”, each one results in exactly one of some fixed finite number k of possible outcomes over n independent trials. ‘x[i]’ indicates the number of times outcome number i was observed over the n trials.

$f(x \mid n, p) = \frac{n!}{\prod_{i=1}^k x_i!} \prod_{i=1}^k p_i^{x_i}$
Parameters: x : (ns, k) int Random variable indicating the number of time outcome i is observed. $$\sum_{i=1}^k x_i=n$$, $$x_i \ge 0$$. n : int Number of trials. p : (k,) Probability of each one of the different outcomes. $$\sum_{i=1}^k p_i = 1)$$, $$p_i \ge 0$$.

Note

• $$E(X_i)=n p_i$$
• $$Var(X_i)=n p_i(1-p_i)$$
• $$Cov(X_i,X_j) = -n p_i p_j$$
• If $$\sum_i p_i < 0.999999$$ a log-likelihood value of -inf

will be returned.

## 9.4. Multivariate continuous distributions¶

pymc.distributions.dirichlet_like(x, theta)[source]

Dirichlet log-likelihood.

This is a multivariate continuous distribution.

$f(\mathbf{x}) = \frac{\Gamma(\sum_{i=1}^k \theta_i)}{\prod \Gamma(\theta_i)}\prod_{i=1}^{k-1} x_i^{\theta_i - 1} \cdot\left(1-\sum_{i=1}^{k-1}x_i\right)^\theta_k$
Parameters: x : (n, k-1) array Array of shape (n, k-1) where n is the number of samples and k the dimension. $$0 < x_i < 1$$, $$\sum_{i=1}^{k-1} x_i < 1$$ theta : array An (n,k) or (1,k) array > 0.

Note

Only the first k-1 elements of x are expected. Can be used as a parent of Multinomial and Categorical nevertheless.

pymc.distributions.mv_normal_like(x, mu, tau)[source]

Multivariate normal log-likelihood

$f(x \mid \pi, T) = \frac{|T|^{1/2}}{(2\pi)^{1/2}} \exp\left\{ -\frac{1}{2} (x-\mu)^{\prime}T(x-\mu) \right\}$
Parameters: x : (n,k) mu : (k) Location parameter sequence. Tau : (k,k) Positive definite precision matrix.
pymc.distributions.mv_normal_chol_like(x, mu, sig)[source]

Multivariate normal log-likelihood.

$f(x \mid \pi, \sigma) = \frac{1}{(2\pi)^{1/2}|\sigma|)} \exp\left\{ -\frac{1}{2} (x-\mu)^{\prime}(\sigma \sigma^{\prime})^{-1}(x-\mu) \right\}$
Parameters: x : (n,k) mu : (k) Location parameter. sigma : (k,k) Lower triangular matrix.
pymc.distributions.mv_normal_cov_like(x, mu, C)[source]

Multivariate normal log-likelihood parameterized by a covariance matrix.

$f(x \mid \pi, C) = \frac{1}{(2\pi|C|)^{1/2}} \exp\left\{ -\frac{1}{2} (x-\mu)^{\prime}C^{-1}(x-\mu) \right\}$
Parameters: x : (n,k) mu : (k) Location parameter. C : (k,k) Positive definite covariance matrix.
pymc.distributions.wishart_like(X, n, Tau)[source]

Wishart log-likelihood.

The Wishart distribution is the probability distribution of the maximum-likelihood estimator (MLE) of the precision matrix of a multivariate normal distribution. If Tau=1, the distribution is identical to the chi-square distribution with n degrees of freedom.

For an alternative parameterization based on $$C=T{-1}$$, see wishart_cov_like.

$f(X \mid n, T) = \frac{{\mid T \mid}^{n/2}{\mid X \mid}^{(n-k-1)/2}}{2^{nk/2} \Gamma_p(n/2)} \exp\left\{ -\frac{1}{2} Tr(TX) \right\}$

where $$k$$ is the rank of X.

Parameters: X : matrix Symmetric, positive definite. n : int Degrees of freedom, > 0. Tau : matrix Symmetric and positive definite

Note

Step method MatrixMetropolis will preserve the symmetry of Wishart variables.

pymc.distributions.wishart_cov_like(X, n, C)[source]

wishart_like(X, n, C)

Wishart log-likelihood. The Wishart distribution is the probability distribution of the maximum-likelihood estimator (MLE) of the covariance matrix of a multivariate normal distribution. If C=1, the distribution is identical to the chi-square distribution with n degrees of freedom.

For an alternative parameterization based on $$T=C^{-1}$$, see wishart_like.

$f(X \mid n, C) = {\mid C^{-1} \mid}^{n/2}{\mid X \mid}^{(n-k-1)/2} \exp\left\{ -\frac{1}{2} Tr(C^{-1}X) \right\}$

where $$k$$ is the rank of X.

Parameters: X : matrix Symmetric, positive definite. n : int Degrees of freedom, > 0. C : matrix Symmetric and positive definite