Simple linear regression is a powerful tool for understanding relationships between variables. It helps us predict one variable based on another, like guessing someone's blood pressure from their age. The method uses a straight line to represent this relationship.
The least squares method finds the best-fitting line by minimizing differences between observed and predicted values. We can use this line to make predictions, but it's important to remember that these predictions aren't perfect and come with some uncertainty.
Simple Linear Regression
Concept of simple linear regression
Top images from around the web for Concept of simple linear regression Linear Regression (2 of 4) | Concepts in Statistics View original
Is this image relevant?
Simple Linear regression algorithm in machine learning with example - Codershood View original
Is this image relevant?
Simple linear regression - Wikipedia View original
Is this image relevant?
Linear Regression (2 of 4) | Concepts in Statistics View original
Is this image relevant?
Simple Linear regression algorithm in machine learning with example - Codershood View original
Is this image relevant?
1 of 3
Top images from around the web for Concept of simple linear regression Linear Regression (2 of 4) | Concepts in Statistics View original
Is this image relevant?
Simple Linear regression algorithm in machine learning with example - Codershood View original
Is this image relevant?
Simple linear regression - Wikipedia View original
Is this image relevant?
Linear Regression (2 of 4) | Concepts in Statistics View original
Is this image relevant?
Simple Linear regression algorithm in machine learning with example - Codershood View original
Is this image relevant?
1 of 3
Statistical method models relationship between two variables
Independent (explanatory) variable: x x x (age, income)
Dependent (response) variable: y y y (blood pressure, spending)
Determines strength and direction of linear relationship between variables
Predicts value of dependent variable based on independent variable
Relationship represented by straight line called regression line
Equation: y = β 0 + β 1 x + ϵ y = \beta_0 + \beta_1x + \epsilon y = β 0 + β 1 x + ϵ
β 0 \beta_0 β 0 : y-intercept , value of y y y when x = 0 x = 0 x = 0
β 1 \beta_1 β 1 : slope , change in y y y for one-unit change in x x x
ϵ \epsilon ϵ : random error term , deviation of observed values from regression line
Least squares method application
Estimates parameters β 0 \beta_0 β 0 and β 1 \beta_1 β 1 of regression line
Minimizes sum of squared differences between observed and predicted y y y values
Estimate slope β 1 \beta_1 β 1 using formula: β 1 ^ = ∑ i = 1 n ( x i − x ˉ ) ( y i − y ˉ ) ∑ i = 1 n ( x i − x ˉ ) 2 \hat{\beta_1} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n} (x_i - \bar{x})^2} β 1 ^ = ∑ i = 1 n ( x i − x ˉ ) 2 ∑ i = 1 n ( x i − x ˉ ) ( y i − y ˉ )
β 1 ^ \hat{\beta_1} β 1 ^ : estimated slope
x i x_i x i , y i y_i y i : observed values of independent and dependent variables
x ˉ \bar{x} x ˉ , y ˉ \bar{y} y ˉ : sample means of independent and dependent variables
n n n : number of observations
Estimate y-intercept β 0 \beta_0 β 0 using formula: β 0 ^ = y ˉ − β 1 ^ x ˉ \hat{\beta_0} = \bar{y} - \hat{\beta_1}\bar{x} β 0 ^ = y ˉ − β 1 ^ x ˉ
β 0 ^ \hat{\beta_0} β 0 ^ : estimated y-intercept
Derivation of regression equation
Substitute estimated values of β 0 \beta_0 β 0 and β 1 \beta_1 β 1 into general equation: y = β 0 + β 1 x + ϵ y = \beta_0 + \beta_1x + \epsilon y = β 0 + β 1 x + ϵ
Resulting regression equation : y ^ = β 0 ^ + β 1 ^ x \hat{y} = \hat{\beta_0} + \hat{\beta_1}x y ^ = β 0 ^ + β 1 ^ x
y ^ \hat{y} y ^ : predicted value of dependent variable for given value of independent variable x x x
Interpretation of regression components
Slope β 1 ^ \hat{\beta_1} β 1 ^ represents change in dependent variable y y y for one-unit change in independent variable x x x
Positive slope indicates positive linear relationship (weight and height)
Negative slope indicates negative linear relationship (age and reaction time)
Y-intercept β 0 ^ \hat{\beta_0} β 0 ^ represents value of dependent variable y y y when independent variable x x x equals zero
May not have meaningful interpretation if range of x x x values does not include zero (negative age)
Predictions using regression equation
Substitute given value of independent variable x x x into equation: y ^ = β 0 ^ + β 1 ^ x \hat{y} = \hat{\beta_0} + \hat{\beta_1}x y ^ = β 0 ^ + β 1 ^ x
Resulting value y ^ \hat{y} y ^ is predicted value of dependent variable for given value of independent variable
Predictions subject to uncertainty
Accuracy depends on strength of linear relationship and quality of model fit (correlation coefficient , residual analysis)