Statistics for AIML - Regression Metrics - Probability Tutorial
What is Probability?
- Probability implies 'likelihood' or 'chance'. When an event is certain to happen then the probability of occurrence of that event is 1 and when it is certain that the event cannot happen then the probability of that event is 0.
- Hence the value of probability ranges from 0 to 1.
Classical Definition of Probability
- As the name suggests the classical approach to defining probability is the oldest approach. It states that if there are n exhaustive, mutually exclusive, and equally likely cases out of which m cases are favorable to the happening of event A,
- Then the probabilities of event A are defined as given by the following probability function:
Example-
Problem Statement:
- A coin is tossed. What is the probability of getting a head?
Solution:
- Number of outcomes favorable to head (m) = 1
- Total number of outcomes (n) = 2 (i.e. head or tail)
PROBABILITY - BASIC CONCEPTS
- Random Experiment
An experiment is said to be a random experiment if its outcome can't be predicted with certainty.
Example
If a coin is tossed, we can't say, whether a head or tail will appear. So it is a random experiment.
- Sample Space
The set of all possible outcomes of an experiment is called the sample space. It is denoted by 'S' and its number of elements is n(s).
Example
In throwing a dice, the number that appears at the top is any one of 1,2,3,4,5,6. So here:
S ={1,2,3,4,5,6} and n(s) = 6
Similarly in the case of a coin, S={Head, Tail} or {H, T} and n(s)=2.
The sample space of rolling 2 dice is 36 i.e.
{(1.1),(1,2),(1,3),(1,4),(1,5),(1,6)}
{(2,1),.........................................}
.
.
{(6,1),..........................................}
- Event
Every subset of a sample space is an event. It is denoted by 'E'.
Example
In throwing a dice S={1,2,3,4,5,6}, the appearance of an even number will be the event E={2,4,6}.
Clearly, E is a subset of S.
1] Equally likely events
Events are said to be equally likely if the probability of occurrence of the events are same.
Example
When a dice is thrown, all the six faces {1,2,3,4,5,6} are equally likely to come up.
2] Exhaustive events
When every possible outcome of an experiment is considered.
Example
A dice is thrown, and cases 1,2,3,4,5,6 form an exhaustive set of events.
3] Mutually exclusive or Disjoint event
If two or more events can't occur simultaneously,
that is no two of them can occur together.
Example
When a coin is tossed, the event of occurrence of a head and the event of occurrence of a tail are mutually exclusive events.
4] Independent or Mutually independent events
Two or more events are said to be independent if the occurrence or non-occurrence of any of them does not affect the probability of occurrence or non-occurrence of the other event.
Example
When a coin is tossed twice, the event of the occurrence of the head in the first throw and the event of the occurrence of the head in the second throw are independent events.
Difference between mutually exclusive and mutually independent events
Mutual exclusiveness is used when the events are taken from the same experiment, whereas independence is used when the events are taken from different experiments.
Mutually Exclusive - Taken from the same experiment and 2 events can't occur simultaneously
E.g. 1. - When a coin is tossed, the event of the occurrence of a head and the event of the occurrence of a tail are mutually exclusive events.
i.e E1 = H and E2 = T
Two sets are known to be mutually exclusive when they have no common elements.
E.g. 2. - When a dice is tossed, the event of the occurrence of an even number and the event of the occurrence of the odd number are mutually exclusive events.
i.e E1 = 2,4,6 and E2 = 1,3,5
Non-Mutually Exclusive OR Mutually Non-Exclusive - Taken from the same experiment and 2 events can happen simultaneously
Two sets are known to be non-mutually exclusive when they have common elements.
E.g. - When a dice is tossed, the event of the occurrence of an even number and the event of the occurrence of numbers less than 3 are non-mutually exclusive events.
i.e E1 = 2,4,6 and E2 = 1,2,3
Mutually Independent - Taken from the different experiments and the outcome or occurrence of one event will not depend on the outcome or occurrence of another event
E.g. 1 - When there are 2 different events i.e. rain is coming and class is ongoing. the outcome or occurrence of both events does not depend on each other.
Mutually dependent - Taken from the different experiments and one event will depend on another event
E.g. 1 - When there are 2 different events i.e. car stops at the signal and the signal is red. both events depend on each other.
E.g. 2 - Consider choosing a card at random from a standard deck of 52 playing cards, and then choosing a second card without replacing the first. The probability that the first card is a queen is 4/52 but the probability that the second card is a jack is 4/51. If the first card chosen is a jack, then the probability of choosing a jack second is 3/51. Here the outcome or occurrence of 1st event will depend on the outcome or occurrence of another event.
ADDITIVE THEOREM OF PROBABILITY
- For Non Mutually Exclusive Events
Statement: If A and B are not mutually exclusive events, the probability of the occurrence of either A or B or both is equal to the probability that event A occurs, plus the probability that event B occurs minus the probability of occurrence of the events common to both A and B. In other words, the probability of occurrence of at least one of them is given by
- For Mutually Exclusive Events
Statement: If A and B are two mutually exclusive events, then the probability of occurrence of either A or B is the sum of the individual probabilities of A and B. Symbolically
ADDITIVE THEOREM OF PROBABILITY - EXAMPLES
- For Non Mutually Exclusive Events
1] A shooter is known to hit a target 3 out of 7 shots; whereas another shooter is known to hit the target 2 out of 5 shots. Find the probability of the target being hit at all when both of them try.
The probability of first shooter hitting the target P (A) = 3/7
The probability of the second shooter hitting the target P (B) = 2/5
Events A and B are not mutually exclusive as both shooters may hit the target. Hence the additive rule is
therefore P( A or B) = P(A) + P(B) - P(A&B)
= 3/7 + 2/5 - (3/7 x 2/5)
= 3/7 + 2/5 - 6/35
= 22/35
2] In a math class of 30 students, 17 are boys and 13 are girls. On a unit test, 4 boys and 5 girls made an A grade. If a student is chosen at random from the class, what is the probability of choosing a girl or an A student?
- For Mutually Exclusive Events
1] A card is drawn from a pack of 52, what is the probability that it is a king or a queen?
Let Event (A) = Draw of a card of king
Event (B) Draw a card of the queen
P (card draw is king or queen) = P (card is king) + P (card is queen)
therefore P( A U B) = P(A) + P(B)
= 4/52 + 4/52
= 2/13
2] A single 6-sided die is rolled. What is the probability of rolling a 2 or a 5?
MULTIPLICATIVE THEOREM OF PROBABILITY
- For Independent Events
Statement: The theorem states that the probability of the simultaneous occurrence of two events that are independent is given by the product of their individual probabilities.
- For Dependent Events (Conditional Probability)
If we recall the dependent event(), the earlier stated multiplicative theorem is not applicable to dependent events. For dependent events, we have another theorem called the conditional probability which is given as:
The probability of event B given event A equals the probability of event A and event B divided by the probability of event A
P(B/A) is the Probability of B if A is already occurred
Example -
Dependent Event:
E.g. 1 - The probability of drawing a king from a bunch of 52 cards is 4/52. The probability of drawing a queen from a bunch of 52 cards is 4/52.
If the king is already drawn, then the probability of drawing the other king is 3/51 and the probability of drawing the queen is 4/51
i.e E1 is Probability of K if K already drawn/occurred P(K | K) = 3/51 and E2 is Probability of Q if K already drawn/occurred P(Q | K) = 4/51
Both Events are dependent on each other
E.g. 2 - Dependent Event: An urn contains 20 red and 10 blue balls. Two balls are drawn from a bag one after the other without replacement. What is the probability that both the balls drawn are red?
Independent Event:
E.g. 1 - You have a cowboy hat, a top hat, and an Indonesian hat called a songkok. You also have four shirts: white, black, green, and pink. If you choose one hat and one shirt at random, what is the probability that you choose the songkok and the black shirt?
The two events are independent events; the choice of hat has no effect on the choice of shirt.
There are three different hats, so the probability of choosing the songkok is 1/3.
There are four different shirts, so the probability of choosing the black shirt is 1/4.
So, by the Multiplication Rule:
P(songkok and black shirt)=(1/3)⋅(1/4)=1/12