1.1 Fundamentals of probability theory

Probability theory lays the foundation for understanding uncertainty and making informed decisions. From basic axioms to advanced techniques like Bayes' theorem, these concepts help us quantify and analyze the likelihood of events in various scenarios.

Mastering probability calculations, conditional probability, and independence is crucial for tackling real-world problems. These tools enable us to update beliefs with new information, model complex relationships, and make predictions in fields ranging from medicine to finance.

Probability Theory Fundamentals

Definition of probability axioms

Top images from around the web for Definition of probability axioms

• 

  Probability for Data Scientists View original

  Is this image relevant?

• 

  Probability axioms - Wikipedia View original

  Is this image relevant?

• 

  Tree diagram (probability theory) - Wikipedia View original

  Is this image relevant?

• 

  Probability for Data Scientists View original

  Is this image relevant?

• 

  Probability axioms - Wikipedia View original

  Is this image relevant?

1 of 3

Top images from around the web for Definition of probability axioms

• 

  Probability for Data Scientists View original

  Is this image relevant?

• 

  Probability axioms - Wikipedia View original

  Is this image relevant?

• 

  Tree diagram (probability theory) - Wikipedia View original

  Is this image relevant?

• 

  Probability for Data Scientists View original

  Is this image relevant?

• 

  Probability axioms - Wikipedia View original

  Is this image relevant?

1 of 3

• Probability measures likelihood of event occurrence ranging from 0 (impossible) to 1 (certain)
• Axioms of probability form foundation:
  • Non-negativity: $P(A) \geq 0$ for any event A ensures no negative probabilities
  • Normalization: $P(S) = 1$, where S is sample space guarantees total probability of all outcomes equals 1
  • Additivity: For mutually exclusive events A and B, $P(A \cup B) = P(A) + P(B)$ allows combining probabilities
• Sample space encompasses all possible outcomes in experiment (dice roll: 1-6)
• Event represents subset of sample space (rolling even number: 2, 4, 6)

Methods for probability calculation

• Counting techniques simplify complex probability calculations:
  • Multiplication principle used for independent events multiplies individual event outcomes (flipping coin twice: 2 x 2 = 4 outcomes)
  • Permutations calculate ordered arrangements of objects using formula $P(n,r) = \frac{n!}{(n-r)!}$ (arranging 5 books on shelf)
  • Combinations determine unordered selections of objects with formula $C(n,r) = \frac{n!}{r!(n-r)!}$ (selecting 3 students from class of 30)
• Complement rule finds probability of event not occurring $P(A') = 1 - P(A)$ (probability of not rolling a 6)
• Addition rule for non-mutually exclusive events: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ (probability of drawing a heart or face card)
• Probability distributions describe likelihood of outcomes:
  • Discrete: probability mass function (rolling dice)
  • Continuous: probability density function (height of population)

Conditional probability and independence

• Conditional probability calculates likelihood of event given another event occurred using $P(A|B) = \frac{P(A \cap B)}{P(B)}$ (probability of rain given cloudy sky)
• Independence exists when $P(A|B) = P(A)$, meaning events don't influence each other (coin flips)
• Multiplication rule for dependent events: $P(A \cap B) = P(A) \cdot P(B|A)$ (drawing two cards without replacement)
• Law of total probability breaks down complex events: $P(A) = P(A|B_1)P(B_1) + P(A|B_2)P(B_2) + ... + P(A|B_n)P(B_n)$ (probability of passing exam considering different study times)

Applications of Bayes' theorem

• Bayes' theorem updates probabilities with new information: $P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}$
• Components include:
  • Prior probability $P(A)$: initial belief (prevalence of disease)
  • Likelihood $P(B|A)$: probability of evidence given hypothesis (test accuracy)
  • Evidence $P(B)$: probability of observing data (positive test result)
  • Posterior probability $P(A|B)$: updated belief after considering evidence
• Applications span various fields:
  • Medical diagnosis updates disease probability based on test results
  • Spam filtering determines email legitimacy using word frequencies
  • Fraud detection in banking transactions
• Bayesian inference uses observed data to refine probability distributions (estimating coin fairness after multiple flips)
• Bayesian networks model complex probabilistic relationships graphically (genetic inheritance patterns)