11.2 Probability theory and Bayesian inference

Probability theory and Bayesian inference are crucial tools in inductive logic. They help us quantify uncertainty and update our beliefs based on new evidence. This topic builds on earlier concepts of logical reasoning, extending them to handle real-world scenarios with incomplete information.

Understanding probability and Bayes' theorem allows us to make better decisions under uncertainty. We'll explore how to calculate probabilities, combine them using rules, and update our beliefs as new information comes in. This knowledge is essential for critical thinking in many fields.

Fundamentals of probability theory

Basic concepts and definitions

Top images from around the web for Basic concepts and definitions

• 

  Why It Matters: Inference for One Proportion | Concepts in Statistics View original

  Is this image relevant?

• 

  Bayes' theorem - Wikipedia View original

  Is this image relevant?

• 

  Bayesian Probability Illustration Diagram | TikZ example View original

  Is this image relevant?

• 

  Why It Matters: Inference for One Proportion | Concepts in Statistics View original

  Is this image relevant?

• 

  Bayes' theorem - Wikipedia View original

  Is this image relevant?

1 of 3

Top images from around the web for Basic concepts and definitions

• 

  Why It Matters: Inference for One Proportion | Concepts in Statistics View original

  Is this image relevant?

• 

  Bayes' theorem - Wikipedia View original

  Is this image relevant?

• 

  Bayesian Probability Illustration Diagram | TikZ example View original

  Is this image relevant?

• 

  Why It Matters: Inference for One Proportion | Concepts in Statistics View original

  Is this image relevant?

• 

  Bayes' theorem - Wikipedia View original

  Is this image relevant?

1 of 3

• Probability measures the likelihood that an event will occur expressed as a number between 0 and 1
  • 0 indicates impossibility (flipping a coin and getting neither heads nor tails)
  • 1 indicates certainty (the sun will rise tomorrow)
• The probability of an event A is denoted as P(A) calculated by dividing the number of favorable outcomes by the total number of possible outcomes assuming all outcomes are equally likely
  • Example: Rolling a fair six-sided die, the probability of getting a 3 is P(3) = 1/6
• The complement of an event A, denoted as A' or ¬A, is the event that A does not occur
  • Its probability is given by P(A') = 1 - P(A)
  • Example: If the probability of a coin landing heads is 0.5, the probability of not getting heads (landing tails) is 1 - 0.5 = 0.5

Rules for combining probabilities

• The addition rule states that for two events A and B, the probability of either A or B occurring is given by P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
  • P(A ∩ B) is the probability of both A and B occurring simultaneously (the intersection)
  • Example: The probability of drawing a heart or a king from a standard deck of cards is P(heart) + P(king) - P(heart and king) = 13/52 + 4/52 - 1/52 = 16/52
• The multiplication rule states that for two events A and B, the probability of both A and B occurring is given by P(A ∩ B) = P(A) × P(B|A)
  • P(B|A) is the conditional probability of B given that A has occurred
  • Example: The probability of drawing a king and then a heart from a standard deck of cards (without replacement) is P(king) × P(heart|king) = 4/52 × 12/51 = 48/2652
• Two events A and B are considered independent if the occurrence of one does not affect the probability of the other
  • Their joint probability is given by P(A ∩ B) = P(A) × P(B)
  • Example: The probability of flipping a coin and getting heads, then rolling a die and getting a 6, is 0.5 × 1/6 = 1/12 (the events are independent)

Bayes' Theorem for updating probabilities

Bayes' theorem formula and components

• Bayes' theorem is a mathematical formula used to calculate the conditional probability of an event based on prior knowledge and new evidence
• The theorem states that P(A|B) = (P(B|A) × P(A)) / P(B)
  • P(A|B) is the probability of event A given that event B has occurred (posterior probability)
  • P(B|A) is the probability of event B given that event A has occurred (likelihood)
  • P(A) is the prior probability of event A
  • P(B) is the probability of event B (marginal probability)
• Example: In a medical test for a disease, Bayes' theorem can be used to calculate the probability that a person has the disease given a positive test result

Updating probabilities with new evidence

• Bayes' theorem allows for the updating of probabilities as new evidence becomes available making it a powerful tool for inductive reasoning and decision-making under uncertainty
• The prior probability represents the initial belief or knowledge about the likelihood of an event before considering any new evidence
  • Example: The prior probability of a person having a rare disease might be based on the prevalence of the disease in the general population
• The likelihood function quantifies the probability of observing the new evidence given each possible hypothesis or event
  • Example: The likelihood of a positive test result given that a person has the disease might be based on the sensitivity of the test
• The marginal probability of the evidence is calculated by summing the product of the prior probabilities and the likelihoods for all possible hypotheses or events
  • Example: The marginal probability of a positive test result is the sum of the probability of a true positive (person has the disease and tests positive) and a false positive (person does not have the disease but tests positive)

Conditional probabilities and inductive reasoning

Understanding conditional probabilities

• Conditional probability is the probability of an event A occurring given that another event B has already occurred, denoted as P(A|B)
  • Example: The probability of drawing a red card given that a face card has been drawn from a standard deck of cards is P(red|face card) = 6/12 = 1/2
• Conditional probabilities are essential for understanding the relationship between events and how the occurrence of one event may influence the likelihood of another event
  • Example: The probability of a person having a positive test result for a disease is influenced by whether the person actually has the disease
• In inductive reasoning, conditional probabilities help to update beliefs or hypotheses based on new evidence or information
  • Example: If a person tests positive for a rare disease, the probability that they actually have the disease (given the positive test result) can be calculated using Bayes' theorem and the prior probability of the disease in the population

Conditional independence

• Two events A and B are considered conditionally independent given a third event C if P(A ∩ B|C) = P(A|C) × P(B|C)
  • This means that once the condition C is known, the occurrence of A does not provide any additional information about the occurrence of B, and vice versa
  • Example: The probability of a person having a positive test result for a disease and experiencing specific symptoms might be conditionally independent given the person's age, as age could influence both the test result and the presence of symptoms
• Understanding conditional probabilities is crucial for making accurate predictions, decisions, and judgments in various domains, such as medical diagnosis, machine learning, and scientific reasoning
  • Example: In medical diagnosis, conditional probabilities can help determine the likelihood of a patient having a specific disease based on their symptoms, test results, and other relevant factors

Fallacies and biases in probabilistic reasoning

Common fallacies in probability

• The base rate fallacy occurs when individuals ignore or underweight the base rate (prior probability) of an event and instead focus on specific information or evidence that may be less relevant or reliable
  • Example: Overestimating the probability of a rare disease based on a positive test result while ignoring the low prevalence of the disease in the population
• The conjunction fallacy is the belief that the probability of two events occurring together (conjunction) is higher than the probability of either event occurring alone, violating the axioms of probability theory
  • Example: Believing that the probability of a person being both a bank teller and a feminist is higher than the probability of them being a bank teller alone
• The gambler's fallacy is the belief that if a particular event occurs more frequently than expected in the past, it is less likely to occur in the future (or vice versa), assuming that events are dependent when they are actually independent
  • Example: Believing that after a series of heads in coin flips, tails are more likely to occur next, despite each flip being independent

Cognitive biases affecting probability judgments

• Confirmation bias is the tendency to search for, interpret, or recall information in a way that confirms one's preexisting beliefs or hypotheses while giving less attention to information that contradicts it
  • Example: Focusing on evidence that supports a favored hypothesis while dismissing evidence that challenges it
• The availability heuristic is a mental shortcut that relies on immediate examples or information that come to mind when evaluating the probability or frequency of an event, leading to biased judgments
  • Example: Overestimating the probability of plane crashes because they are more readily available in memory due to media coverage
• Overcoming these fallacies and biases requires a solid understanding of probability theory, critical thinking, and the ability to recognize and counteract cognitive biases in reasoning and decision-making
  • Example: Deliberately seeking out evidence that challenges one's beliefs and considering alternative hypotheses to mitigate confirmation bias