📈Intro to Probability for Business Unit 11 – Simple Linear Regression & Correlation

Key Concepts

• Simple linear regression models the relationship between two quantitative variables using a linear equation
• Correlation coefficient measures the strength and direction of the linear relationship between two variables
• Least squares method estimates the regression line by minimizing the sum of squared residuals
• Residuals represent the differences between observed values and predicted values from the regression line
• Coefficient of determination (R-squared) measures the proportion of variability in the response variable explained by the predictor variable
• Hypothesis testing and confidence intervals assess the significance and precision of the estimated regression coefficients
• Assumptions of simple linear regression include linearity, independence, normality, and constant variance of residuals

Linear Relationships

• A linear relationship between two quantitative variables implies a constant rate of change (slope) between them
• Scatterplots visually represent the relationship between two variables, with each point representing an observation
  • A positive linear relationship shows an upward trend (e.g., as years of experience increase, salary tends to increase)
  • A negative linear relationship shows a downward trend (e.g., as price increases, demand tends to decrease)
• The strength of a linear relationship is indicated by how closely the points in a scatterplot follow a straight line
• Outliers are observations that deviate substantially from the overall pattern and can influence the linear relationship
• Correlation does not imply causation; other factors may be responsible for the observed relationship

Correlation Coefficient

• The correlation coefficient (r) quantifies the strength and direction of the linear relationship between two variables
• r ranges from -1 to +1, with 0 indicating no linear relationship
  • Positive r values indicate a positive linear relationship (e.g., r = 0.8 suggests a strong positive linear relationship)
  • Negative r values indicate a negative linear relationship (e.g., r = -0.5 suggests a moderate negative linear relationship)
• The square of the correlation coefficient (r-squared) represents the proportion of variability in one variable explained by the other
• Sample correlation coefficient is an estimate of the population correlation coefficient
• Hypothesis tests and confidence intervals can be used to make inferences about the population correlation coefficient

Simple Linear Regression Model

• The simple linear regression model is expressed as: $y = \beta_0 + \beta_1x + \epsilon$
  • y is the response (dependent) variable
  • x is the predictor (independent) variable
  • $\beta_0$ is the y-intercept (value of y when x = 0)
  • $\beta_1$ is the slope (change in y for a one-unit increase in x)
  • $\epsilon$ represents the random error term
• The regression coefficients ($\beta_0$ and $\beta_1$) are unknown population parameters estimated from sample data
• The fitted regression line is used to make predictions for the response variable given values of the predictor variable
• The regression equation allows for interpolation (predictions within the range of observed data) and extrapolation (predictions beyond the range of observed data)

Least Squares Method

• The least squares method estimates the regression coefficients by minimizing the sum of squared residuals
• Residuals are the differences between the observed y values and the predicted y values from the regression line
• The least squares estimates for the regression coefficients are:
  • $\hat{\beta}_1 = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}$
  • $\hat{\beta}_0 = \bar{y} - \hat{\beta}_1\bar{x}$
• The least squares method provides unbiased estimates of the population regression coefficients
• The least squares regression line always passes through the point $(\bar{x}, \bar{y})$

Interpreting Regression Results

• The estimated slope ($\hat{\beta}_1$) represents the change in the mean response for a one-unit increase in the predictor variable
• The estimated y-intercept ($\hat{\beta}_0$) represents the mean response when the predictor variable equals zero
• Hypothesis tests (t-tests) and confidence intervals can be used to assess the significance and precision of the estimated regression coefficients
• The coefficient of determination (R-squared) measures the proportion of variability in the response variable explained by the predictor variable
  • R-squared ranges from 0 to 1, with higher values indicating a better fit of the regression line to the data
• The standard error of the estimate measures the average distance between the observed values and the predicted values from the regression line

Model Assumptions and Diagnostics

• Simple linear regression relies on several assumptions for valid inference and prediction:
  • Linearity: The relationship between the response and predictor variables is linear
  • Independence: The observations are independent of each other
  • Normality: The residuals follow a normal distribution with mean zero
  • Constant variance (homoscedasticity): The variability of the residuals is constant across all levels of the predictor variable
• Residual plots can be used to assess the validity of these assumptions
  • Residuals vs. fitted values plot checks for linearity and constant variance
  • Normal probability plot checks for normality of residuals
• Outliers and influential observations can be identified using leverage, studentized residuals, and Cook's distance
• Violations of assumptions may require data transformations or alternative regression methods

Applications in Business

• Simple linear regression is widely used in business for modeling and predicting relationships between variables
• Examples of applications include:
  • Predicting sales based on advertising expenditure
  • Analyzing the relationship between customer satisfaction and loyalty
  • Estimating production costs based on the number of units produced
  • Forecasting demand for a product based on its price
• Regression results can inform business decisions, such as setting prices, allocating resources, and evaluating marketing strategies
• Caution should be exercised when interpreting regression results, as correlation does not imply causation, and other factors may influence the relationship between variables