The Origin of Bayes Theorem
Table of Contents
I'm reading "The theory that would not die" and these are notes I took from them. The book didn't really give me a clear idea about what Price's argument was so I also read a Quartz article about that part of the story and, of course, Wikipedia came into it at some points.
A Brief Sketch of The Timelines to Bayes' Theorem
- 1718: Abraham de Moivre publishes The Doctrine of Chances, the first textbook on probability.
- 1746-1749: Somewhere in this period Thomas Bayes comes writes An Essay towards solving a Problem in the Doctrine of the Chances which describes elements of Inverse Probability, in which the probability of a cause is calculated based on observed effects, stated as a thought experiment in which a person turned away from a table estimates the position of a ball based on being told whether subsequent balls randomly dropped on the same table are to the left or the right of it.
- 1748: David Hume publishes Of Miracles, in which he argues that since miracles are, by nature, singular, they can never have as much evidence in their favor as against them.
- 1749: Pierre-Simon Laplace is born
- 1764: Richard Price publishes An Essay towards solving a Problem in the Doctrine of the Chances with his additions, believing that it could act as a refutation of Hume's argument
- 1774: Laplace comes up with idea that the probability of a cause given the observed effect is the ratio of the probability of that effect given the cause to sum of the probabilities for all other causes given that effect.
- 1781: Price tells the Marquis of Condorcet about Bayes' work and Laplace incorporates the use of the prior into his formulation
- 1810: Laplace discovers the Central Limit Theorem
- 1814: Laplace extends his version of Bayes' equation
The Equations
Since it's hard to write out the equations in bullet points I'm going to write some simple versions here.
Bayes' Formulation
"The theory that would not die" notes that Bayes' didn't write out an equation, but it can be written out something like this. \[ P(\textit{cause}|\textit{effect}) = \frac{P(\textit{effect}|\textit{cause}) P(\textit{cause})}{P(\textit{effect})} \]
Laplace's First Version
Originally Laplace didn't have the prior's in his equation (I'll substitute C for cause, E for effect and C' for not our theorized cause). \[ P(C|E) = \frac{P(E|C)}{\sum P(E|C')} \]
Laplace's Final Version
\[ P(C|E) = \frac{P(E|C)P_{\textit{prior}}(C)}{\sum P(E|C') P_{\textit{prior}} (C')} \]
Sources
- McGrayne SB. The theory that would not die: how Bayes’ rule cracked the enigma code, hunted down Russian submarines, and emerged triumphant from two centuries of controversy. paperback ed. New Haven, Conn.: Yale University Press; 2011. 336 p.
- Kopf D. The most important formula in data science was first used to prove the existence of God [Internet]. Quartz. [cited 2019 May 12]. Available from: https://qz.com/1315731/the-most-important-formula-in-data-science-was-first-used-to-prove-the-existence-of-god/