Uncertainty is Normal

Probability provides the language and tools to handle uncertainty.

Probability of an Event

- Event (A): An outcome to which a probability is assigned.
- Sample Space (S): The set of possible outcomes or events.
- Probability Function (P): The function used to assign a probability to an event.

Probability distribution: Shape or distribution of all events in the sample space

Two Schools of Probability

  • Frequentist Probability: objective approach
    • Events are observed and counted, and their frequencies provide the basis for directly calculating a probability.
    • p-values
    • confidence intervals

Probability theory was originally developed to analyze the frequencies of events.

  • Bayesian Probability: subjective approach
    • Probabilities are assigned to events based on evidence and personal belief and are centered around Bayes' theorem.
    • Bayes factors
    • credible interval for inference
    • Bayes estimator
    • maximum estimation for parameter estimation

One big advantage of Bayesian interpretation is that it can because to model our uncertainty about events that do not have long-term frequencies.

Joint, Marginal, and Conditional Probability

For two random variables $(A, B)$, probability of event $X = A$ and $Y = B$ equals

  • Joint Probability: Probability of events A and B; $P(A \ and \ B) = P(A\cap B)= P(X=A, Y=B)$
  • Marginal Probability: Probability of event A given variable Y; $P(X = A) = \sum_{i \in Y} P(X=A, Y=i)$
  • Conditional Probability: Probability of event A given event B; $\displaystyle P(X=A|Y=B) = \frac{P(X=A, Y=B)}{P(Y=B)}$

Probability for Indepencdence and Exclusivity

  • Independent $X$ and $Y$
    • Joint: $P(X=A \cap Y=B) = P(X=A) P(Y=B)$
    • Marginal: $P(X=A, Y) = P(X=A)$
    • Conditional: $P(X=A|Y=B) = P(X=A)$
  • Mutually exclusive $X$ and $Y$
    • Joint: $P(X=A \cap Y=B) = 0
    • $P(A \ or \ B) = P(A) + P(B) = P(A \cup B)$