5.4 Covariance and correlation

Covariance and correlation are key concepts in understanding relationships between random variables. They measure how two variables change together, with covariance indicating joint variability and correlation providing a standardized measure of linear dependence.

These tools are crucial for analyzing joint probability distributions. By quantifying the strength and direction of relationships between variables, they help us interpret complex data and make predictions about how changes in one variable might affect another.

Covariance and Correlation of Variables

Defining Covariance and Correlation

Top images from around the web for Defining Covariance and Correlation

• 

  Covariance - Wikipedia View original

  Is this image relevant?

• 

  Covariance - Wikipedia View original

  Is this image relevant?

• 

  CSUP Math 156 Correlation and Linear Regression View original

  Is this image relevant?

• 

  Covariance - Wikipedia View original

  Is this image relevant?

• 

  Covariance - Wikipedia View original

  Is this image relevant?

1 of 3

Top images from around the web for Defining Covariance and Correlation

• 

  Covariance - Wikipedia View original

  Is this image relevant?

• 

  Covariance - Wikipedia View original

  Is this image relevant?

• 

  CSUP Math 156 Correlation and Linear Regression View original

  Is this image relevant?

• 

  Covariance - Wikipedia View original

  Is this image relevant?

• 

  Covariance - Wikipedia View original

  Is this image relevant?

1 of 3

• Covariance measures the joint variability of two random variables
  • Indicates how much they change together
  • Calculated as the expected value of the product of their deviations from their respective means
• Correlation is a standardized version of covariance, ranging from -1 to 1
  • Measures the strength and direction of the linear relationship between two random variables
  • More easily interpretable due to its standardized scale
• Both covariance and correlation measure the dependence between two random variables

Interpreting the Sign and Magnitude

• The sign of the covariance or correlation indicates the direction of the relationship
  • Positive values indicate a direct relationship (variables tend to increase or decrease together)
  • Negative values indicate an inverse relationship (one variable tends to increase as the other decreases)
• The magnitude of the correlation, but not the covariance, indicates the strength of the linear relationship between the variables
  • Correlation of ±1 indicates a perfect linear relationship
  • Correlation of 0 indicates no linear relationship

Calculating Covariance and Correlation

Discrete Random Variables

• The covariance of two discrete random variables X and Y can be calculated using their joint probability distribution
  • Formula: $Cov(X,Y) = \Sigma(x - \mu_X)(y - \mu_Y) * P(X=x, Y=y)$
  • $\mu_X$ and $\mu_Y$ are the means of X and Y, respectively
• The correlation coefficient ($\rho$) can be calculated by dividing the covariance by the product of the standard deviations of X and Y
  • Formula: $\rho(X,Y) = Cov(X,Y) / (\sigma_X * \sigma_Y)$

Continuous Random Variables and Sample Statistics

• For continuous random variables, the sums in the covariance and correlation formulas are replaced by double integrals over the joint probability density function
• The sample covariance and correlation can be calculated using the same formulas
  • Replace the true means and standard deviations with their sample counterparts
  • Replace the probabilities with the observed frequencies

Interpreting Covariance and Correlation

Interpreting the Values

• A positive covariance or correlation indicates that the two variables tend to increase or decrease together
• A negative value indicates that one variable tends to increase as the other decreases
• A covariance or correlation of zero suggests that there is no linear relationship between the variables
  • Does not necessarily imply independence

Comparing Covariance and Correlation

• The magnitude of the covariance is difficult to interpret directly
  • Depends on the scales of the variables
• The correlation coefficient is standardized and ranges from -1 to 1
  • ±1 indicates a perfect linear relationship
  • 0 indicates no linear relationship
• The square of the correlation coefficient ($R^2$) represents the proportion of the variance in one variable that can be explained by the linear relationship with the other variable

Independence vs Zero Covariance/Correlation

Independence Implies Zero Covariance/Correlation

• If two random variables are independent, their covariance and correlation will always be zero
  • Independence implies that the variables have no relationship with each other

Zero Covariance/Correlation Does Not Imply Independence

• A covariance or correlation of zero does not necessarily imply that the variables are independent
  • Only indicates that there is no linear relationship between them
• Non-linear relationships between variables can exist even when the covariance or correlation is zero
  • Example: a quadratic relationship ($Y = X^2$) has a correlation of zero but is clearly not independent

Establishing Independence

• To establish independence, one must show that the joint probability distribution of the variables is equal to the product of their marginal distributions for all possible values of the variables
  • $P(X,Y) = P(X) * P(Y)$ for all $x$ and $y$
• Independence is a stronger condition than zero covariance or correlation
  • Requires that the variables have no relationship of any kind, not just a lack of linear relationship