Bayesian statistics

Bayes' theorem is used in Bayesian methods to update probabilities, which are degrees of belief, after obtaining new data. Given two events ${\displaystyle A}$ and ${\displaystyle B}$, the conditional probability of ${\displaystyle A}$ given that ${\displaystyle B}$ is true is expressed as follows:[7]

$${\displaystyle P(A\mid B)={\frac {P(B\mid A)P(A)}{P(B)}}}$$

where ${\displaystyle P(B)\neq 0}$. Although Bayes' theorem is a fundamental result of probability theory, it has a specific interpretation in Bayesian statistics. In the above equation, ${\displaystyle A}$ usually represents a proposition (such as the statement that a coin lands on heads fifty percent of the time) and ${\displaystyle B}$ represents the evidence, or new data that is to be taken into account (such as the result of a series of coin flips). ${\displaystyle P(A)}$ is the prior probability of ${\displaystyle A}$ which expresses one's beliefs about ${\displaystyle A}$ before evidence is taken into account. The prior probability may also quantify prior knowledge or information about ${\displaystyle A}$. ${\displaystyle P(B\mid A)}$ is the likelihood function, which can be interpreted as the probability of the evidence ${\displaystyle B}$ given that ${\displaystyle A}$ is true. The likelihood quantifies the extent to which the evidence ${\displaystyle B}$ supports the proposition ${\displaystyle A}$. ${\displaystyle P(A\mid B)}$ is the posterior probability, the probability of the proposition ${\displaystyle A}$ after taking the evidence ${\displaystyle B}$ into account. Essentially, Bayes' theorem updates one's prior beliefs ${\displaystyle P(A)}$ after considering the new evidence ${\displaystyle B}$.[2]

The probability of the evidence ${\displaystyle P(B)}$ can be calculated using the law of total probability. If ${\displaystyle \{A_{1},A_{2},\dots ,A_{n}\}}$ is a partition of the sample space, which is the set of all outcomes of an experiment, then,[2][7]

$${\displaystyle P(B)=P(B\mid A_{1})P(A_{1})+P(B\mid A_{2})P(A_{2})+\dots +P(B\mid A_{n})P(A_{n})=\sum _{i}P(B\mid A_{i})P(A_{i})}$$

When there are an infinite number of outcomes, it is necessary to integrate over all outcomes to calculate ${\displaystyle P(B)}$ using the law of total probability. Often, ${\displaystyle P(B)}$ is difficult to calculate as the calculation would involve sums or integrals that would be time-consuming to evaluate, so often only the product of the prior and likelihood is considered, since the evidence does not change in the same analysis. The posterior is proportional to this product:[2]

$${\displaystyle P(A\mid B)\propto P(B\mid A)P(A)}$$

The maximum a posteriori, which is the mode of the posterior and is often computed in Bayesian statistics using mathematical optimization methods, remains the same. The posterior can be approximated even without computing the exact value of ${\displaystyle P(B)}$ with methods such as Markov chain Monte Carlo or variational Bayesian methods.[2]

The general set of statistical techniques can be divided into a number of activities, many of which have special Bayesian versions.

• Bayesian epistemology
• For a list of mathematical logic notation used in this article
  • Notation in probability and statistics
  • List of logic symbols

• Bernardo, José M.; Smith, Adrian F. M. (2000). Bayesian Theory. New York: Wiley. ISBN 0-471-92416-4.
• Bolstad, William M.; Curran, James M. (2016). Introduction to Bayesian Statistics (3rd ed.). Wiley. ISBN 978-1-118-09156-2.
• Downey, Allen B. (2021). Think Bayes: Bayesian Statistics in Python (2nd ed.). O'Reilly. ISBN 978-1-4920-8946-9.
• Hoff, Peter D. (2009). A First Course in Bayesian Statistical Methods (2nd ed.). New York: Springer. ISBN 978-1-4419-2828-3.
• Lee, Peter M. (2012). Bayesian Statistics: An Introduction (4th ed.). Wiley. ISBN 978-1-118-33257-3.
• Robert, Christian P. (2007). The Bayesian Choice : From Decision-Theoretic Foundations to Computational Implementation (2nd ed.). New York: Springer. ISBN 978-0-387-71598-8.
• Johnson, Alicia A.; Ott, Miles Q.; Dogucu, Mine. (2022) Bayes Rules! An Introduction to Applied Bayesian Modeling. Chapman and Hall ISBN 9780367255398

Wikiversity has learning resources about Bayesian statistics

• Theo Kypraios. "A Gentle Tutorial in Bayesian Statistics" (PDF). Retrieved 2013-11-03.
• Jordi Vallverdu. Bayesians Versus Frequentists A Philosophical Debate on Statistical Reasoning.
• Bayesian statistics David Spiegelhalter, Kenneth Rice Scholarpedia 4(8):5230. doi:10.4249/scholarpedia.5230
• Bayesian modeling book and examples available for downloading.
• Rens van de Schoot. "A Gentle Introduction to Bayesian Analysis" (PDF).
• Bayesian A/B Testing Calculator Dynamic Yield