8.2 Covariance and correlation

Covariance and correlation measure how two variables change together. Covariance shows if they move in the same or opposite directions, while correlation tells us how strong that relationship is on a scale from -1 to 1.

These concepts help us understand connections between things like height and weight or income and education. They're useful in finance, science, and social research to spot patterns and make predictions about related variables.

Covariance and Correlation

Definition of covariance and correlation

Top images from around the web for Definition of covariance and correlation

• 

  Pearson correlation coefficient - Wikipedia View original

  Is this image relevant?

• 

  CSUP Math 156 Correlation and Linear Regression View original

  Is this image relevant?

• 

  Covariance - Wikipedia View original

  Is this image relevant?

• 

  Pearson correlation coefficient - Wikipedia View original

  Is this image relevant?

• 

  CSUP Math 156 Correlation and Linear Regression View original

  Is this image relevant?

1 of 3

Top images from around the web for Definition of covariance and correlation

• 

  Pearson correlation coefficient - Wikipedia View original

  Is this image relevant?

• 

  CSUP Math 156 Correlation and Linear Regression View original

  Is this image relevant?

• 

  Covariance - Wikipedia View original

  Is this image relevant?

• 

  Pearson correlation coefficient - Wikipedia View original

  Is this image relevant?

• 

  CSUP Math 156 Correlation and Linear Regression View original

  Is this image relevant?

1 of 3

• Covariance quantifies the joint variability of two random variables from their individual means
  • Positive covariance indicates variables tend to move in the same direction relative to their means (height and weight)
  • Negative covariance indicates variables tend to move in opposite directions relative to their means (price and demand)
• Covariance formula: $Cov(X,Y) = E[(X - \mu_X)(Y - \mu_Y)]$
  • $\mu_X$ and $\mu_Y$ represent the means of random variables $X$ and $Y$
• Correlation measures the strength and direction of the linear relationship between two random variables
  • Ranges from -1 (perfect negative linear relationship) to 1 (perfect positive linear relationship)
  • Correlation of 0 implies no linear relationship (income and favorite color)
• Correlation formula: $\rho_{XY} = \frac{Cov(X,Y)}{\sigma_X \sigma_Y}$
  • $\sigma_X$ and $\sigma_Y$ represent the standard deviations of random variables $X$ and $Y$

Calculation of joint distributions

• Covariance calculation: $Cov(X,Y) = E[XY] - E[X]E[Y]$
  • $E[XY]$ represents the expected value of the product of $X$ and $Y$
  • $E[X]$ and $E[Y]$ represent the individual expected values (means) of $X$ and $Y$
• Correlation calculation: $\rho_{XY} = \frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}}$
  • $Var(X)$ and $Var(Y)$ represent the variances of random variables $X$ and $Y$
• For discrete random variables, calculate expected values using the probability mass function (PMF)
  • Example: Roll two fair dice, let $X$ be the sum and $Y$ be the product of the numbers rolled
• For continuous random variables, calculate expected values using the joint probability density function (PDF)
  • Example: $X$ and $Y$ represent the heights of a randomly selected male and female student

Properties of statistical relationships

• Covariance properties
  • $Cov(X,X) = Var(X)$, covariance of a variable with itself equals its variance
  • $Cov(X,Y) = Cov(Y,X)$, covariance is symmetric
  • $Cov(aX + b, cY + d) = ac \cdot Cov(X,Y)$ for constants $a$, $b$, $c$, and $d$
• Correlation properties
  • $\rho_{XX} = 1$, a variable is perfectly correlated with itself
  • $\rho_{XY} = \rho_{YX}$, correlation is symmetric
  • $|\rho_{XY}| \leq 1$, correlation is bounded between -1 and 1
• Relationship between independence and covariance/correlation
  • If $X$ and $Y$ are independent, then $Cov(X,Y) = 0$ and $\rho_{XY} = 0$
  • However, $Cov(X,Y) = 0$ or $\rho_{XY} = 0$ does not necessarily imply independence (non-linear relationships)

Applications in linear analysis

• Interpret covariance and correlation values
  1. Determine the direction of the linear relationship (positive or negative)
  2. Assess the strength of the linear relationship (magnitude of correlation)
• Covariance interpretation is scale-dependent and difficult to compare across different variable pairs
• Correlation provides a standardized measure of linear relationship strength for easier comparison
• Applications across various fields
  • Finance: Portfolio risk analysis and diversification (stocks and bonds)
  • Signal processing: Assessing similarity between signals (audio and video)
  • Machine learning: Feature selection and dimensionality reduction (customer preferences)
  • Social sciences: Studying relationships between variables (education and income)