Baye’s Theorem: Definition, Theorem, Derivation & Examples

Baye’s Theorem is a mathematical formula for determining conditional probability, named after \({18^{{\rm{th}}}}\)-century British mathematician Thomas Baye’s. The possibility of an outcome occurring based on the likelihood of a preceding outcome occurring is known as conditional probability.

Baye’s Theorem can be used to alter previous forecasts or theories from fresh evidence. It is used in the fields like management, chemistry, business, finance, medicine etc., in order to predict the best among the groups. In this article, we will provide detailed information about Baye’s theorem, its definition, derivation, etc. Scroll down to learn more about Baye’s Theorem.

What is Baye’s Theorem?

Baye’s Theorem defines the probability of an event based on prior knowledge of conditions that may be relevant to the event in probability theory and statistics. For example, if it is known that the chance of acquiring health problems increases with age, Baye’s Theorem allows the danger to an individual of a known age to be estimated more correctly. This Theorem is basically based on conditional probability.

For two events, Baye’s Theorem is stated mathematically with the following formula:

 \(P(A\mid B) = \frac{{P(B\mid A)P(A)}}{{P(B)}}\)  Where \(A\) and \(B\) are events in consideration and \(P(B) \ne 0\), where \(P(A\mid B)\) is a conditional probability, i.e. the probability of event \(A\) occurring when event \(B\) already happened, it is also known as the posterior probability of \(A\) given \(B\). \(P(A)\) and \(P(B)\) are the probabilities of observing \(A\) and \(B\) respectively, without any given conditions; they are known as the prior probability or marginal probability.

Consider two bags \(I\) and \(II\).

Bag \(I\): \(2\) white and \(3\) red balls

Bag \(II\): \(4\) white and \(5\) red balls.

One ball is drawn at random from any bag. We can determine the probability of selecting any of the bags (i.e. \(\frac{1}{2}\) Or the probability of drawing a ball of a specific colour (say red) from a specific bag (say Bag \(II\)). So, we can find the probability that the ball is drawn is of a specific colour if from a known bag the ball is drawn. But, can we determine the probability that the ball is drawn from a specific bag (say Bag \(I\)) if the colour of the ball drawn is known? This type of problem can be easily solved by Baye’s Theorem.

Baye’s Theorem Statement

Before stating Baye’s Theorem, we must know about the Theorem on total probability.

Theorem on total probability: Let \({ {E_1},\,{E_2}, \ldots ,\,{E_n}} \) be a partition of the sample space \(S\), and suppose that each of the events \({E_1},\,{E_2}, \ldots ,\,{E_n}\) has the non-zero occurrence of probability. Let \(A\) be an event associated with \(S\), then

\(P(A) = P\left( {{E_1}} \right)P\left( {A\mid {E_1}} \right) + P\left( {{E_2}} \right)P\left( {A\mid {E_2}} \right) + \ldots + P\left( {{E_n}} \right)P\left( {A\mid {E_n}} \right) = \sum\limits_{j = 1}^n P\left( {{E_j}} \right)P\left( {A\mid {E_j}} \right)\)

Baye’s Theorem states that if \(E,\,{E_2}, \ldots ,\,{E_n}\) are \(n\) non-empty events which constitute a partition of sample space \(S\), i.e. \({E_1},\,{E_2}, \ldots ,\,{E_n}\) are pairwise disjoint and \({E_1} \cup {E_2} \cup \ldots \cup {E_n} = S\) and \(A\) is any event of non-zero probability, then

\(P\left( {{E_i}\mid A} \right) = \frac{{P\left( {{E_i}} \right)P\left( {A\mid {E_i}} \right)}}{{\sum\limits_{j = 1}^n P\left( {{E_j}} \right)P\left( {A\mid {E_j}} \right)}}\,{\rm{for}}\,{\rm{any}}\,i = 1,2,3, \ldots ,n\)

Here, the events \({E_1},\,{E_2}, \ldots ,\,{E_n}\) are called hypotheses. The probability \(P(E)\) is called the priori probability of hypothesis \(E\). The conditional probability \(P\left( {{E_i}\mid A} \right)\) is called \(a\) posteriori probability of the hypothesis \({{E_i}}\).

Derivation of Baye’s Theorem

From conditional probability, we know that,

\(P\left( {{E_i}\mid A} \right) = \frac{{P\left( {A \cap {E_i}} \right)}}{{P(A)}}\)

\(P\left( {{E_i}\mid A} \right) = \frac{{P\left( {{E_i}} \right)P\left( {A\mid {E_i}} \right)}}{{P(A)}}\) (by multiplication rule of probability)
\(P\left( {{E_i}\mid A} \right) = \frac{{P\left( {{E_i}} \right)P\left( {A\mid {E_i}} \right)}}{{\sum\limits_{j = 1}^n P \left( {{E_j}} \right)P\left( {A\mid {E_j}} \right)}}\) (by the result of the Theorem of total probability)

Baye’s Theorem Example

Let’s understand Baye’s Theorem with a real-life example.

Suppose, In a group of \(400\) persons, \(160\) are non-vegetarian smokers, \(100\) are vegetarian smokers, and the rest are non-smokers and vegetarians. The chances of contracting a certain chest condition are \({\rm{35\% , 20\% }}\), and \({\rm{10\% }}\), respectively. A person is chosen at random from the group and discovered to be infected with the disease. What is the probability that the chosen person is a non-vegetarian and a smoker?

Let \(A,\,{E_1},\,{E_2}\) and \({E_3}\) represent the events that the person suffers from the disease, is a smoker & a non-vegetarian, is a smoker & a vegetarian, and the person is a non-smoker & a vegetarian, respectively.

\(\therefore P\left( {{E_1}} \right) = \frac{{160}}{{400}},\,P\left( {{E_2}} \right) = \frac{{100}}{{400}},\,P\left( {{E_3}} \right) = \frac{{140}}{{400}}\)

Now,

\(P\left( {\frac{A}{{{E_1}}}} \right) = \frac{{35}}{{100}},\,P\left( {\frac{A}{{{E_2}}}} \right) = \frac{{20}}{{100}},\,P\left( {\frac{A}{{{E_3}}}} \right) = \frac{{10}}{{100}}\)

Using Baye’s Theorem, we get,

Required probability \( = P\left( {\frac{{{E_1}}}{A}} \right) = \frac{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right)}}{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right) + P\left( {{E_2}} \right)P\left( {\frac{A}{{{E_2}}}} \right) + P\left( {{E_3}} \right)P\left( {\frac{A}{{{E_3}}}} \right)}}\)
\( = \frac{{\frac{{160}}{{400}} \times \frac{{35}}{{100}}}}{{\frac{{164}}{{400}} \times \frac{{35}}{{100}} + \frac{{140}}{{400}} \times \frac{{20}}{{100}} + \frac{{140}}{{400}} \times \frac{{10}}{{100}}}}\)
\( = \frac{{560}}{{560 + 200 + 140}} = \frac{{560}}{{900}} = \frac{{28}}{{45}}\)

So, the probability that the selected person is a smoker and non-vegetarian is \(\frac{{28}}{{45}}\)

Applications of Baye’s Theorem

The applications of Baye’s theorem are as follows:

1. One of the significant applications is Bayesian inference. It is defined as the approach to statistical inference. It has various applications like Medicine, Science, Philosophy, sports, etc.
2. One can use Baye’s Theorem to define the accuracy of medical test results.
3. Baye’s theorem depends on consolidating prior probability to develop posterior probabilities.

Solved Examples on Baye’s Theorem

Q.1. A bag \(A\) contains \(2\) white and \(3\) red balls, and a bag \(B\) contains \(4\) white and \(5\) red balls. One ball is drawn at random and is found to be red. Determine the probability that it was drawn from bag \(B\).
Ans: Let \(A,\,{E_1}\) and \({E_2}\) represent the events that the ball is red, \(A\) is chosen, and bag \(B\) is chosen, respectively:
therefore \(P\left( {{E_1}} \right) = \frac{1}{2},\,P\left( {{E_2}} \right) = \frac{1}{2}\)
Now, \(P\left( {A/{E_1}} \right) = \frac{3}{5}\)
\(P\left( {A/{E_2}} \right) = \frac{5}{9}\)
Using Baye’s Theorem, we get the required probability \( = P\left( {{E_2}/A} \right) = \frac{{P\left( {{E_2}} \right)P\left( {A/{E_2}} \right)}}{{P\left( {{E_1}} \right)P\left( {A/{E_1}} \right) + P\left( {{E_2}} \right)P\left( {A/{E_2}} \right)}}\)
\( = \frac{{\frac{1}{2} \times \frac{5}{9}}}{{\frac{1}{2} \times \frac{3}{5} + \frac{1}{2} \times \frac{5}{9}}} = \frac{{25}}{{52}}\)

Q.2.Three urns contain \(2\) white and \(3\) black balls,\(3\) white and \(2\) black balls and \(4\) white and \(1\) black ball, respectively. One ball is drawn from an urn chosen at random, and it is known to be white. Determine the probability that it was drawn from the \({{\rm{1}}^{{\rm{st}}}}\) urn.
Ans: Let \({E_1},\,{E_2}\) and \({E_3}\) represent the events of selecting Urn \(I\), Urn \(II\) and Urn \(III\), respectively.
Let \(A\) be the event that the ball drawn is white. \(\therefore \,P\left( {{E_1}} \right) = \frac{1}{3}\)
\(P\left( {{E_2}} \right) = \frac{1}{3}, P\left( {{E_3}} \right) = \frac{1}{3}\)
Now, \(P\left( {A/{E_1}} \right) = \frac{2}{5}\)
\(P\left( {A/{E_2}} \right) = \frac{3}{5}, P\left( {A/{E_3}} \right) = \frac{4}{5}\)

Using Baye’s Theorem, we get,
Required probability
\(P\left( {{E_1}/A} \right) = \frac{{P\left( {{E_1}} \right)P\left( {A/{E_1}} \right)}}{{P\left( {{E_1}} \right)P\left( {A/{E_1}} \right) + P\left( {{E_2}} \right)P\left( {A/{E_2}} \right) + P\left( {{E_3}} \right)P\left( {A/{E_3}} \right)}}\)
\( = \frac{{\frac{1}{3} \times \frac{2}{5}}}{{\frac{1}{3} \times \frac{2}{5} + \frac{1}{3} \times \frac{3}{5} + \frac{1}{3} \times \frac{4}{5}}}\)
\( = \frac{2}{{2 + 3 + 4}} = \frac{2}{9}\)

Q.3. Suppose \(5\) men out of \(100\) , and \(25\) women out of \(1000\) are good orators. A random orator is chosen. Calculate the probability of a man being chosen. Assume that the number of males and women is equal.
Ans: \(A,\,{E_1}\) and \({E_2}\) denote the events that the person is a good orator, is a man and is a woman, respectively.
Let
\(\therefore \,P\left( {{E_1}} \right) = \frac{1}{2}, P\left( {{E_2}} \right) = \frac{1}{2}\)
Now,
\(P\left( {\frac{A}{{{E_1}}}} \right) = \frac{5}{{100}}\)
\(P\left( {\frac{A}{{{E_2}}}} \right) = \frac{{25}}{{1000}}\)

Using Baye’s Theorem, we get,
Required probability \( = P\left( {\frac{{{E_1}}}{A}} \right) = \frac{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right)}}{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right) + P\left( {{E_2}} \right)P\left( {\frac{A}{{{E_2}}}} \right)}}\)
\( = \frac{{\frac{1}{2} \times \frac{5}{{100}}}}{{\frac{1}{2} \times \frac{5}{{100}} + \frac{1}{2} \times \frac{{25}}{{1000}}}}\)
\( = \frac{1}{{1 + \frac{1}{2}}} = \frac{2}{3}\)

Q.4.A letter has been identified as coming from either LONDON or CLIFTON. Only two consecutive letters ON are visible on the envelope. What are the chances that the letter came from LONDON?
Ans: \(A,\,{E_1}\) and \({E_2}\) denote the events that the two consecutive letters are visible, the letter has come from LONDON, and the letter has come from CLIFTON, respectively.
\(\therefore \,P\left( {{E_1}} \right) = \frac{1}{2}, P\left( {{E_2}} \right) = \frac{1}{2}\)
Now,
\(P\left( {\frac{A}{{{E_1}}}} \right) = \frac{2}{5}, P\left( {\frac{A}{{{E_2}}}} \right) = \frac{1}{6}\)
Using Baye’s Theorem, we get,
Required probability \( = P\left( {\frac{{{E_1}}}{A}} \right) = \frac{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right)}}{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right) + P\left( {{E_2}} \right)P\left( {\frac{A}{{{E_2}}}} \right)}}\)
\( = \frac{{\frac{1}{2} \times \frac{2}{5}}}{{\frac{1}{2} \times \frac{2}{5} + \frac{1}{2} \times \frac{1}{6}}} = \frac{{\frac{2}{5}}}{{\frac{2}{5} + \frac{1}{6}}} = \frac{{\frac{2}{5}}}{{\frac{{17}}{{30}}}} = \frac{{12}}{{17}}\)

Q.5. An insurance company insured \(3000\) scooters, \(4000\) cars and \(5000\) trucks. The probabilities of an accident involving a scooter, a car and a truck are \(0.02,\,0.03\), and \(0.04\) respectively. One of the insured vehicles met with an accident. Find the probability that it was a scooter.
Ans: Let \({E_1},\,{E_2}\) and \({E_3}\) denote the events that the vehicle is a scooter, a car and a truck, respectively.
Let \(A\) be the event that the vehicle met with an accident.
It is given that there are \(3000\) scooters, \(4000\) cars and \(5000\) trucks.
Total number of vehicles \( = 3000 + 4000 + 5000 = 12000\)
\(P\left( {{E_1}} \right) = \frac{{3000}}{{12000}} = \frac{1}{4}, P\left( {{E_2}} \right) = \frac{{4000}}{{12000}} = \frac{1}{3}, P\left( {{E_3}} \right) = \frac{{5000}}{{12000}} = \frac{5}{{12}}\)
Now,
\(P\left( {\frac{A}{{{E_1}}}} \right) = 0.02 = \frac{2}{{100}}, P\left( {\frac{A}{{{E_2}}}} \right) = 0.03 = \frac{3}{{100}}, P\left( {\frac{A}{{{E_3}}}} \right) = 0.04 = \frac{4}{{100}}\)
The probability that the vehicle, which met with an accident, is a scooter is given by \(P\left( {\frac{{{E_1}}}{A}} \right)\).

Using Baye’s Theorem, we get:
Required probability \( = P\left( {\frac{{{E_1}}}{A}} \right) = \frac{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right)}}{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right) + P\left( {{E_2}} \right)P\left( {\frac{A}{{{E_2}}}} \right) + + P\left( {{E_2}} \right)P\left( {\frac{A}{{{E_2}}}} \right)}}\)
\( = \frac{{\frac{1}{4} \times \frac{2}{{100}}}}{{\frac{1}{4} \times \frac{2}{{100}} + \frac{1}{3} \times \frac{3}{{100}} + \frac{5}{{12}} \times \frac{4}{{100}}}} = \frac{{\frac{1}{2}}}{{\frac{1}{2} + 1 + \frac{3}{3}}} = \frac{{\frac{1}{2}}}{{\frac{{3 + 6 + 10}}{6}}} = \frac{3}{{19}}\)

Q.6. In a group of \(400\) persons, \(160\) are non-vegetarian smokers, \(100\) are vegetarian smokers, and the rest are non-smokers and vegetarians. The chances of contracting a certain chest condition are \(35\% ,\,20\% \), and \(10\% \), respectively. A person is chosen at random from the group and discovered to be infected with the disease. What is the probability that the chosen person is a non-vegetarian and a smoker?
Ans: Let \(A,\,{E_1},\,{E_2}\) and \({E_3}\) represent the events that the person suffers from the disease, is a smoker & a non-vegetarian, is a smoker & a vegetarian and the person is a non-smoker & a vegetarian, respectively.
\(\therefore P\left( {{E_1}} \right) = \frac{{160}}{{400}}, P\left( {{E_2}} \right) = \frac{{100}}{{400}}, P\left( {{E_3}} \right) = \frac{{140}}{{400}}\)
Now,
\(P\left( {\frac{A}{{{E_1}}}} \right) = \frac{{35}}{{100}}, P\left( {\frac{A}{{{E_2}}}} \right) = \frac{{20}}{{100}}, P\left( {\frac{A}{{{E_3}}}} \right) = \frac{{10}}{{100}}\)

Using Baye’s Theorem, we get:
Required probability \( = P\left( {\frac{{{E_1}}}{A}} \right) = \frac{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right)}}{{P\left( {{E_1}} \right)P\left( {\frac{A}{{{E_1}}}} \right) + P\left( {{E_2}} \right)P\left( {\frac{A}{{{E_2}}}} \right) + P\left( {{E_3}} \right)P\left( {\frac{A}{{{E_3}}}} \right)}}\)
\( = \frac{{\frac{{160}}{{400}} \times \frac{{35}}{{100}}}}{{\frac{{164}}{{400}} \times \frac{{35}}{{100}} + \frac{{140}}{{400}} \times \frac{{20}}{{100}} + \frac{{140}}{{400}} \times \frac{{10}}{{100}}}}\)
\( = \frac{{560}}{{560 + 200 + 140}} = \frac{{560}}{{900}} = \frac{{28}}{{45}}\)
So, the probability that the selected person is a smoker and non-vegetarian is \(\frac{{28}}{{45}}\)

Summary

Baye’s Theorem is a popular way to calculate conditional probabilities. It is employed in the computation of posterior probabilities. We have learnt about Baye’s Theorem, derivations and solved real-life examples. It is used in the field of medical science, manufacturing industries, business, Financial market, machine learning etc. It offers a consistent method of learning from data for both probabilistic parametric inference and probabilistic model comparison. It has a wide range of real-world applications, making it an important part of mathematics.

FAQs on Baye’s Theorem

Q.1. How do you calculate Baye’s Theorem?
Ans: The formula used to calculate Baye’s Theorem is \(P\left( {{E_i}\mid A} \right) = \frac{{P\left( {{E_i}} \right)P\left( {A\mid {E_i}} \right)}}{{\sum\limits_{j = 1}^n P\left( {{E_j}} \right)P\left( {A\mid {E_j}} \right)}}\) for any \(i = 1,\,2,\,3, \ldots ,\,n\)

Q.2. What is Baye’s Theorem, and when can it be used?
Ans: Baye’s Theorem describes the probability of an event based on prior knowledge of conditions that may be relevant to the event in probability theory and statistics. It is a special case of conditional probability. It is used in the field of medical science, manufacturing industries, business, Financial market, machine learning etc., to predict the best among the groups.

Q.3. Where can Baye’s Theorem be used?
Ans: Baye’s rule can be used in the following examples:
Drug testing: To find out someone has taken drugs or not
Cancer rate: To predict cancer or any disease as per the symptoms of the individuals
Defective item rate: To find the rate of the defective item from the given set of machines
Stock market rate: To find out the crash of a particular stock during the fall of its index.

Q.4. Is Baye’s Theorem always true?
Ans: The Baye’s Rule is a result that derives from conditional probability. It is therefore always true as the conditional probability is always true.

Q.5. What does Baye’s theorem state?
Ans: Baye’s Theorem states that If \(E,\,{E_2}, \ldots ,\,{E_n}\) are \(n\) non-empty events which constitute a partition of sample space \(S\), i.e. \(E,\,{E_2}, \ldots ,\,{E_n}\) are pairwise disjoint and \({E_1} \cup {E_2} \cup \ldots \cup {E_n} = S\) and \(A\) is any event of non-zero probability, then
\(P\left( {{E_i}\mid A} \right) = \frac{{P\left( {{E_i}} \right)P\left( {A\mid {E_i}} \right)}}{{\sum\limits_{j = 1}^n P\left( {{E_j}} \right)P\left( {A\mid {E_j}} \right)}}\) for any \(i = 1,\,2,\,3,\, \ldots ,\,n\)
Here, the events \({E_1},\,{E_2}, \ldots ,\,{E_n}\) are called hypotheses. The probability \(P(E)\) is called the priori probability of hypothesis \(E\). The conditional probability \(P\left( {{E_i}\mid A} \right)\) is called a posteriori probability of the hypothesis \({{E_i}}\).