9. Probability distributions

PyMC provides a large suite of built-in probability distributions. For each distribution, it provides:

  • A function that evaluates its log-probability or log-density: normal_like().
  • A function that draws random variables: rnormal().
  • A function that computes the expectation associated with the distribution: normal_expval().
  • A Stochastic subclass generated from the distribution: Normal.

This section describes the likelihood functions of these distributions.

9.1. Discrete distributions

pymc.distributions.bernoulli_like(x, p)[source]

Bernoulli log-likelihood

The Bernoulli distribution describes the probability of successes (x=1) and failures (x=0).

\[f(x \mid p) = p^{x} (1-p)^{1-x}\]
Parameters:
  • x : Series of successes (1) and failures (0). \(x=0,1\)
  • p : Probability of success. \(0 < p < 1\).
Example:
>>> from pymc import bernoulli_like
>>> bernoulli_like([0,1,0,1], .4)
-2.854232711280291

Note

  • \(E(x)= p\)
  • \(Var(x)= p(1-p)\)
pymc.distributions.binomial_like(x, n, p)[source]

Binomial log-likelihood. The discrete probability distribution of the number of successes in a sequence of n independent yes/no experiments, each of which yields success with probability p.

\[f(x \mid n, p) = \frac{n!}{x!(n-x)!} p^x (1-p)^{n-x}\]
Parameters:
  • x : [int] Number of successes, > 0.
  • n : [int] Number of Bernoulli trials, > x.
  • p : Probability of success in each trial, \(p \in [0,1]\).

Note

  • \(E(X)=np\)
  • \(Var(X)=np(1-p)\)
pymc.distributions.categorical_like(x, p)[source]

Categorical log-likelihood. The most general discrete distribution.

\[f(x=i \mid p) = p_i\]

for \(i \in 0 \ldots k-1\).

Parameters:
  • x : [int] \(x \in 0\ldots k-1\)
  • p : [float] \(p > 0\), \(\sum p = 1\)
pymc.distributions.discrete_uniform_like(x, lower, upper)[source]

Discrete uniform log-likelihood.

\[f(x \mid lower, upper) = \frac{1}{upper-lower}\]
Parameters:
  • x : [int] \(lower \leq x \leq upper\)
  • lower : Lower limit.
  • upper : Upper limit (upper > lower).
pymc.distributions.geometric_like(x, p)[source]

Geometric log-likelihood. The probability that the first success in a sequence of Bernoulli trials occurs on the x’th trial.

\[f(x \mid p) = p(1-p)^{x-1}\]
Parameters:
  • x : [int] Number of trials before first success (x > 0).
  • p : Probability of success on an individual trial, \(p \in [0,1]\)

Note

  • \(E(X)=1/p\)
  • \(Var(X)=\frac{1-p}{p^2}\)
pymc.distributions.hypergeometric_like(x, n, m, N)[source]

Hypergeometric log-likelihood.

Discrete probability distribution that describes the number of successes in a sequence of draws from a finite population without replacement.

\[\begin{split}f(x \mid n, m, N) = \frac{\left({ \begin{array}{c} {m} \\ {x} \\ \end{array} }\right)\left({ \begin{array}{c} {N-m} \\ {n-x} \\ \end{array}}\right)}{\left({ \begin{array}{c} {N} \\ {n} \\ \end{array}}\right)}\end{split}\]
Parameters:
  • x : [int] Number of successes in a sample drawn from a population.
  • n : [int] Size of sample drawn from the population.
  • m : [int] Number of successes in the population.
  • N : [int] Total number of units in the population.

Note

\(E(X) = \frac{n n}{N}\)

pymc.distributions.negative_binomial_like(x, mu, alpha)[source]

Negative binomial log-likelihood.

The negative binomial distribution describes a Poisson random variable whose rate parameter is gamma distributed. PyMC’s chosen parameterization is based on this mixture interpretation.

\[f(x \mid \mu, \alpha) = \frac{\Gamma(x+\alpha)}{x! \Gamma(\alpha)} (\alpha/(\mu+\alpha))^\alpha (\mu/(\mu+\alpha))^x\]
Parameters:
  • x : x = 0,1,2,...
  • mu : mu > 0
  • alpha : alpha > 0

Note

  • \(E[x]=\mu\)
  • In Wikipedia’s parameterization, \(r=\alpha\) \(p=\alpha/(\mu+\alpha)\) \(\mu=rp/(1-p)\)
pymc.distributions.poisson_like(x, mu)[source]

Poisson log-likelihood.

The Poisson is a discrete probability distribution. It is often used to model the number of events occurring in a fixed period of time when the times at which events occur are independent. The Poisson distribution can be derived as a limiting case of the binomial distribution.

\[f(x \mid \mu) = \frac{e^{-\mu}\mu^x}{x!}\]
Parameters:
  • x : [int] \(x \in {0,1,2,...}\)
  • mu : Expected number of occurrences during the given interval, \(\mu \geq 0\).

Note

  • \(E(x)=\mu\)
  • \(Var(x)=\mu\)
pymc.distributions.truncated_poisson_like(x, mu, k)[source]

Truncated Poisson log-likelihood.

The Truncated Poisson is a discrete probability distribution that is arbitrarily truncated to be greater than some minimum value k. For example, zero-truncated Poisson distributions can be used to model counts that are constrained to be non-negative.

\[f(x \mid \mu, k) = \frac{e^{-\mu}\mu^x}{x!(1-F(k|\mu))}\]
Parameters:
  • x : [int] \(x \in {0,1,2,...}\)

  • mu : Expected number of occurrences during the given interval,

    \(\mu \geq 0\).

  • k : Truncation point representing the minimum allowable value.

Note

  • \(E(x)=\frac{\mu}{1-F(k|\mu)}\)
  • \(Var(x)=\frac{\mu}{1-F(k|\mu)}\)

9.2. Continuous distributions

pymc.distributions.beta_like(x, alpha, beta)[source]

Beta log-likelihood. The conjugate prior for the parameter \(p\) of the binomial distribution.

\[f(x \mid \alpha, \beta) = \frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha) \Gamma(\beta)} x^{\alpha - 1} (1 - x)^{\beta - 1}\]
Parameters:
  • x : 0 < x < 1
  • alpha : alpha > 0
  • beta : beta > 0
Example:
>>> from pymc import beta_like
>>> beta_like(.4,1,2)
0.182321556793954

Note

  • \(E(X)=\frac{\alpha}{\alpha+\beta}\)
  • \(Var(X)=\frac{\alpha \beta}{(\alpha+\beta)^2(\alpha+\beta+1)}\)
pymc.distributions.cauchy_like(x, alpha, beta)[source]

Cauchy log-likelihood. The Cauchy distribution is also known as the Lorentz or the Breit-Wigner distribution.

\[f(x \mid \alpha, \beta) = \frac{1}{\pi \beta [1 + (\frac{x-\alpha}{\beta})^2]}\]
Parameters:
  • alpha : Location parameter.
  • beta : Scale parameter > 0.

Note

  • Mode and median are at alpha.
pymc.distributions.chi2_like(x, nu)[source]

Chi-squared \(\chi^2\) log-likelihood.

\[f(x \mid \nu) = \frac{x^{(\nu-2)/2}e^{-x/2}}{2^{\nu/2}\Gamma(\nu/2)}\]
Parameters:
  • x : > 0
  • nu : [int] Degrees of freedom ( nu > 0 )

Note

  • \(E(X)=\nu\)
  • \(Var(X)=2\nu\)
pymc.distributions.degenerate_like(x, k)[source]

Degenerate log-likelihood.

\[\begin{split}f(x \mid k) = \left\{ \begin{matrix} 1 \text{ if } x = k \\ 0 \text{ if } x \ne k\end{matrix} \right.\end{split}\]
Parameters:
  • x : Input value.
  • k : Degenerate value.
pymc.distributions.exponential_like(x, beta)[source]

Exponential log-likelihood.

The exponential distribution is a special case of the gamma distribution with alpha=1. It often describes the time until an event.

\[f(x \mid \beta) = \beta e^{-\beta x}\]
Parameters:
  • x : x > 0
  • beta : Rate parameter (beta > 0).

Note

  • \(E(X) = 1/\beta\)
  • \(Var(X) = 1/\beta^2\)
  • PyMC’s beta is named ‘lambda’ by Wikipedia, SciPy, Wolfram MathWorld and other sources.
pymc.distributions.exponweib_like(x, alpha, k, loc=0, scale=1)[source]

Exponentiated Weibull log-likelihood.

The exponentiated Weibull distribution is a generalization of the Weibull family. Its value lies in being able to model monotone and non-monotone failure rates.

\[\begin{split}f(x \mid \alpha,k,loc,scale) & = \frac{\alpha k}{scale} (1-e^{-z^k})^{\alpha-1} e^{-z^k} z^{k-1} \\ z & = \frac{x-loc}{scale}\end{split}\]
Parameters:
  • x : x > 0
  • alpha : Shape parameter
  • k : k > 0
  • loc : Location parameter
  • scale : Scale parameter (scale > 0).
pymc.distributions.gamma_like(x, alpha, beta)[source]

Gamma log-likelihood.

Represents the sum of alpha exponentially distributed random variables, each of which has rate parameter beta.

\[f(x \mid \alpha, \beta) = \frac{\beta^{\alpha}x^{\alpha-1}e^{-\beta x}}{\Gamma(\alpha)}\]
Parameters:
  • x : math:x ge 0
  • alpha : Shape parameter (alpha > 0).
  • beta : Rate parameter (beta > 0).

Note

  • \(E(X) = \frac{\alpha}{\beta}\)
  • \(Var(X) = \frac{\alpha}{\beta^2}\)
pymc.distributions.half_cauchy_like(x, alpha, beta)[source]

Half-Cauchy log-likelihood. Simply the absolute value of Cauchy.

\[f(x \mid \alpha, \beta) = \frac{2}{\pi \beta [1 + (\frac{x-\alpha}{\beta})^2]}\]
Parameters:
  • alpha : Location parameter.
  • beta : Scale parameter (beta > 0).

Note

  • x must be non-negative.
pymc.distributions.half_normal_like(x, tau)[source]

Half-normal log-likelihood, a normal distribution with mean 0 limited to the domain \(x \in [0, \infty)\).

\[f(x \mid \tau) = \sqrt{\frac{2\tau}{\pi}}\exp\left\{ {\frac{-x^2 \tau}{2}}\right\}\]
Parameters:
  • x : \(x \ge 0\)
  • tau : tau > 0
pymc.distributions.inverse_gamma_like(x, alpha, beta)[source]

Inverse gamma log-likelihood, the reciprocal of the gamma distribution.

\[f(x \mid \alpha, \beta) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} x^{-\alpha - 1} \exp\left(\frac{-\beta}{x}\right)\]
Parameters:
  • x : x > 0
  • alpha : Shape parameter (alpha > 0).
  • beta : Scale parameter (beta > 0).

Note

\(E(X)=\frac{\beta}{\alpha-1}\) for \(\alpha > 1\) \(Var(X)=\frac{\beta^2}{(\alpha-1)^2(\alpha-2)}\) for \(\alpha > 2\)

pymc.distributions.laplace_like(x, mu, tau)[source]

Laplace (double exponential) log-likelihood.

The Laplace (or double exponential) distribution describes the difference between two independent, identically distributed exponential events. It is often used as a heavier-tailed alternative to the normal.

\[f(x \mid \mu, \tau) = \frac{\tau}{2}e^{-\tau |x-\mu|}\]
Parameters:
  • x : \(-\infty < x < \infty\)
  • mu : Location parameter \(-\infty < mu < \infty\)
  • tau : Scale parameter \(\tau > 0\)

Note

  • \(E(X) = \mu\)
  • \(Var(X) = \frac{2}{\tau^2}\)
pymc.distributions.logistic_like(x, mu, tau)[source]

Logistic log-likelihood.

The logistic distribution is often used as a growth model; for example, populations, markets. Resembles a heavy-tailed normal distribution.

\[f(x \mid \mu, tau) = \frac{\tau \exp(-\tau[x-\mu])}{[1 + \exp(-\tau[x-\mu])]^2}\]
Parameters:
  • x : \(-\infty < x < \infty\)
  • mu : Location parameter \(-\infty < mu < \infty\)
  • tau : Scale parameter (tau > 0)

Note

  • \(E(X) = \mu\)
  • \(Var(X) = \frac{\pi^2}{3\tau^2}\)
pymc.distributions.lognormal_like(x, mu, tau)[source]

Log-normal log-likelihood.

Distribution of any random variable whose logarithm is normally distributed. A variable might be modeled as log-normal if it can be thought of as the multiplicative product of many small independent factors.

\[f(x \mid \mu, \tau) = \sqrt{\frac{\tau}{2\pi}}\frac{ \exp\left\{ -\frac{\tau}{2} (\ln(x)-\mu)^2 \right\}}{x}\]
Parameters:
  • x : x > 0
  • mu : Location parameter.
  • tau : Scale parameter (tau > 0).

Note

\(E(X)=e^{\mu+\frac{1}{2\tau}}\) \(Var(X)=(e^{1/\tau}-1)e^{2\mu+\frac{1}{\tau}}\)

pymc.distributions.noncentral_t_like(x, mu, lam, nu)[source]

Non-central Student’s T log-likelihood.

Describes a normal variable whose precision is gamma distributed.

\[f(x|\mu,\lambda,\nu) = \frac{\Gamma(\frac{\nu + 1}{2})}{\Gamma(\frac{\nu}{2})} \left(\frac{\lambda}{\pi\nu}\right)^{\frac{1}{2}} \left[1+\frac{\lambda(x-\mu)^2}{\nu}\right]^{-\frac{\nu+1}{2}}\]
Parameters:
  • x : Input data.
  • mu : Location parameter.
  • lambda : Scale parameter.
  • nu : Degrees of freedom.
pymc.distributions.normal_like(x, mu, tau)[source]

Normal log-likelihood.

\[f(x \mid \mu, \tau) = \sqrt{\frac{\tau}{2\pi}} \exp\left\{ -\frac{\tau}{2} (x-\mu)^2 \right\}\]
Parameters:
  • x : Input data.
  • mu : Mean of the distribution.
  • tau : Precision of the distribution, which corresponds to \(1/\sigma^2\) (tau > 0).

Note

  • \(E(X) = \mu\)
  • \(Var(X) = 1/\tau\)
pymc.distributions.pareto_like(x, alpha, m)[source]

Pareto log-likelihood. The Pareto is a continuous, positive probability distribution with two parameters. It is often used to characterize wealth distribution, or other examples of the 80/20 rule.

\[f(x \mid \alpha, m) = \frac{\alpha m^{\alpha}}{x^{\alpha+1}}\]
Parameters:
  • x : Input data (x > m)
  • alpha : Shape parameter (alpha>0)
  • m : Scale parameter (m>0)

Note

  • \(E(x)=\frac{\alpha m}{\alpha-1} if \alpha > 1\)
  • \(Var(x)=\frac{m^2 \alpha}{(\alpha-1)^2(\alpha-2)} if \alpha > 2\)
pymc.distributions.skew_normal_like(x, mu, tau, alpha)[source]

Azzalini’s skew-normal log-likelihood

\[f(x \mid \mu, \tau, \alpha) = 2 \Phi((x-\mu)\sqrt{\tau}\alpha) \phi(x,\mu,\tau)\]

where :math:Phi is the normal CDF and :math: phi is the normal PDF.

Parameters:
  • x : Input data.
  • mu : Mean of the distribution.
  • tau : Precision of the distribution (> 0).
  • alpha : Shape parameter of the distribution.
pymc.distributions.t_like(x, nu)[source]

Student’s T log-likelihood.

Describes a zero-mean normal variable whose precision is gamma distributed. Alternatively, describes the mean of several zero-mean normal random variables divided by their sample standard deviation.

\[f(x \mid \nu) = \frac{\Gamma(\frac{\nu+1}{2})}{\Gamma(\frac{\nu}{2}) \sqrt{\nu\pi}} \left( 1 + \frac{x^2}{\nu} \right)^{-\frac{\nu+1}{2}}\]
Parameters:
  • x : Input data.
  • nu : Degrees of freedom.
pymc.distributions.truncated_normal_like(x, mu, tau, a=None, b=None)[source]

Truncated normal log-likelihood.

\[f(x \mid \mu, \tau, a, b) = \frac{\phi(\frac{x-\mu}{\sigma})} {\Phi(\frac{b-\mu}{\sigma}) - \Phi(\frac{a-\mu}{\sigma})},\]

where \(\sigma^2=1/\tau\), phi is the standard normal PDF and Phi is the standard normal CDF.

Parameters:
  • x : Input data.
  • mu : Mean of the distribution.
  • tau : Precision of the distribution, which corresponds to 1/sigma**2 (tau > 0).
  • a : Left bound of the distribution.
  • b : Right bound of the distribution.
pymc.distributions.truncated_pareto_like(x, alpha, m, b)[source]

Truncated Pareto log-likelihood. The Pareto is a continuous, positive probability distribution with two parameters. It is often used to characterize wealth distribution, or other examples of the 80/20 rule.

\[f(x \mid \alpha, m, b) = \frac{\alpha m^{\alpha} x^{-\alpha}}{1-(m/b)**{\alpha}}\]
Parameters:
  • x : Input data (x > m)
  • alpha : Shape parameter (alpha>0)
  • m : Scale parameter (m>0)
  • b : Upper bound (b>m)
pymc.distributions.uniform_like(x, lower, upper)[source]

Uniform log-likelihood.

\[f(x \mid lower, upper) = \frac{1}{upper-lower}\]
Parameters:
  • x : \(lower \leq x \leq upper\)
  • lower : Lower limit.
  • upper : Upper limit (upper > lower).
pymc.distributions.von_mises_like(x, mu, kappa)[source]

von Mises log-likelihood.

\[f(x \mid \mu, k) = \frac{e^{k \cos(x - \mu)}}{2 \pi I_0(k)}\]

where I_0 is the modified Bessel function of order 0.

Parameters:
  • x : Input data.
  • mu : Mean of the distribution.
  • kappa : Dispersion of the distribution

Note

  • \(E(X) = \mu\)
pymc.distributions.weibull_like(x, alpha, beta)[source]

Weibull log-likelihood

\[f(x \mid \alpha, \beta) = \frac{\alpha x^{\alpha - 1} \exp(-(\frac{x}{\beta})^{\alpha})}{\beta^\alpha}\]
Parameters:
  • x : \(x \ge 0\)
  • alpha : alpha > 0
  • beta : beta > 0

Note

  • \(E(x)=\beta \Gamma(1+\frac{1}{\alpha})\)
  • \(Var(x)=\beta^2 \Gamma(1+\frac{2}{\alpha} - \mu^2)\)

9.3. Multivariate discrete distributions

pymc.distributions.multivariate_hypergeometric_like(x, m)[source]

Multivariate hypergeometric log-likelihood

Describes the probability of drawing x[i] elements of the ith category, when the number of items in each category is given by m.

\[\begin{split}\frac{\prod_i \left({ \begin{array}{c} {m_i} \\ {x_i} \\ \end{array}}\right)}{\left({ \begin{array}{c} {N} \\ {n} \\ \end{array}}\right)}\end{split}\]

where \(N = \sum_i m_i\) and \(n = \sum_i x_i\).

Parameters:
  • x : [int sequence] Number of draws from each category, (x < m).
  • m : [int sequence] Number of items in each categoy.
pymc.distributions.multinomial_like(x, n, p)[source]

Multinomial log-likelihood.

Generalization of the binomial distribution, but instead of each trial resulting in “success” or “failure”, each one results in exactly one of some fixed finite number k of possible outcomes over n independent trials. ‘x[i]’ indicates the number of times outcome number i was observed over the n trials.

\[f(x \mid n, p) = \frac{n!}{\prod_{i=1}^k x_i!} \prod_{i=1}^k p_i^{x_i}\]
Parameters:
x : (ns, k) int

Random variable indicating the number of time outcome i is observed. \(\sum_{i=1}^k x_i=n\), \(x_i \ge 0\).

n : int

Number of trials.

p : (k,)

Probability of each one of the different outcomes. \(\sum_{i=1}^k p_i = 1)\), \(p_i \ge 0\).

Note

  • \(E(X_i)=n p_i\)
  • \(Var(X_i)=n p_i(1-p_i)\)
  • \(Cov(X_i,X_j) = -n p_i p_j\)
  • If \(\sum_i p_i < 0.999999\) a log-likelihood value of -inf

will be returned.

9.4. Multivariate continuous distributions

pymc.distributions.dirichlet_like(x, theta)[source]

Dirichlet log-likelihood.

This is a multivariate continuous distribution.

\[f(\mathbf{x}) = \frac{\Gamma(\sum_{i=1}^k \theta_i)}{\prod \Gamma(\theta_i)}\prod_{i=1}^{k-1} x_i^{\theta_i - 1} \cdot\left(1-\sum_{i=1}^{k-1}x_i\right)^\theta_k\]
Parameters:
x : (n, k-1) array

Array of shape (n, k-1) where n is the number of samples and k the dimension. \(0 < x_i < 1\), \(\sum_{i=1}^{k-1} x_i < 1\)

theta : array

An (n,k) or (1,k) array > 0.

Note

Only the first k-1 elements of x are expected. Can be used as a parent of Multinomial and Categorical nevertheless.

pymc.distributions.mv_normal_like(x, mu, tau)[source]

Multivariate normal log-likelihood

\[f(x \mid \pi, T) = \frac{|T|^{1/2}}{(2\pi)^{1/2}} \exp\left\{ -\frac{1}{2} (x-\mu)^{\prime}T(x-\mu) \right\}\]
Parameters:
  • x : (n,k)
  • mu : (k) Location parameter sequence.
  • Tau : (k,k) Positive definite precision matrix.
pymc.distributions.mv_normal_chol_like(x, mu, sig)[source]

Multivariate normal log-likelihood.

\[f(x \mid \pi, \sigma) = \frac{1}{(2\pi)^{1/2}|\sigma|)} \exp\left\{ -\frac{1}{2} (x-\mu)^{\prime}(\sigma \sigma^{\prime})^{-1}(x-\mu) \right\}\]
Parameters:
  • x : (n,k)
  • mu : (k) Location parameter.
  • sigma : (k,k) Lower triangular matrix.
pymc.distributions.mv_normal_cov_like(x, mu, C)[source]

Multivariate normal log-likelihood parameterized by a covariance matrix.

\[f(x \mid \pi, C) = \frac{1}{(2\pi|C|)^{1/2}} \exp\left\{ -\frac{1}{2} (x-\mu)^{\prime}C^{-1}(x-\mu) \right\}\]
Parameters:
  • x : (n,k)
  • mu : (k) Location parameter.
  • C : (k,k) Positive definite covariance matrix.
pymc.distributions.wishart_like(X, n, Tau)[source]

Wishart log-likelihood.

The Wishart distribution is the probability distribution of the maximum-likelihood estimator (MLE) of the precision matrix of a multivariate normal distribution. If Tau=1, the distribution is identical to the chi-square distribution with n degrees of freedom.

For an alternative parameterization based on \(C=T{-1}\), see wishart_cov_like.

\[f(X \mid n, T) = \frac{{\mid T \mid}^{n/2}{\mid X \mid}^{(n-k-1)/2}}{2^{nk/2} \Gamma_p(n/2)} \exp\left\{ -\frac{1}{2} Tr(TX) \right\}\]

where \(k\) is the rank of X.

Parameters:
X : matrix

Symmetric, positive definite.

n : int

Degrees of freedom, > 0.

Tau : matrix

Symmetric and positive definite

Note

Step method MatrixMetropolis will preserve the symmetry of Wishart variables.

pymc.distributions.wishart_cov_like(X, n, C)[source]

wishart_like(X, n, C)

Wishart log-likelihood. The Wishart distribution is the probability distribution of the maximum-likelihood estimator (MLE) of the covariance matrix of a multivariate normal distribution. If C=1, the distribution is identical to the chi-square distribution with n degrees of freedom.

For an alternative parameterization based on \(T=C^{-1}\), see wishart_like.

\[f(X \mid n, C) = {\mid C^{-1} \mid}^{n/2}{\mid X \mid}^{(n-k-1)/2} \exp\left\{ -\frac{1}{2} Tr(C^{-1}X) \right\}\]

where \(k\) is the rank of X.

Parameters:
X : matrix

Symmetric, positive definite.

n : int

Degrees of freedom, > 0.

C : matrix

Symmetric and positive definite