Alternate Form of Regression Equation: Types, Bivariatem Multivariate, Equations

Alternate Form of Regression Equation: Regression means going back or stepping back. Say you have a set of variables in a system under study. Some of these variables are dependent upon others. If these variables are correlated significantly, we can establish a relationship between the dependent and independent variables using the known variables. This process is called regression analysis.

The crux behind regression analysis is using existing data to establish a relationship between variables and then predicting future results or studying past data. Correlation is simply a measure of how strongly two variables are related to each other. So, it is clear that correlation and regression go hand in hand. Here, we will try to find the regression equation without using the correlation coefficient. Continue reading this article to know everything about alternate form of regression equation, types, bivariatem multivariate, equations and more.

Types of Regression

Below are different types of Regression explained in detail.

LEARN EXAM CONCEPTS ON EMBIBE

• As the name suggests, Bivariate regression involves only two variables, one dependent and the other independent.
• When multiple dependent and independent variables are correlated, they are said to follow multivariate regression.

We shall now focus on Bivariate regression.

Types of Bivariate Regression

From the existing data set of the two variables, we can plot a graph that shows the relationship between the two variables. This graph is called a scatter diagram. If these two variables are correlated, then the scatter diagram will look more or less like a curve called the curve of regression.

When this curve is a straight line, it is called the line of regression, and the relationship between the variables is called linear regression.

When this curve is not a straight line, the relationship between the variables is called non-linear regression.

Equation of the Line of Regression

1. Graphical method: The scatter diagram, plotted using the existing data set, can be used to fit a line on the graph. Using the inclination of this line, \(\theta\), and its \(y-\)intercept, \(c\), we can obtain the equation of this line by using the equation \(y = mx + c\).
2. Analytical method: We can also fit a line using the existing data set by the method of least squares, which minimises the sum of squares of deviations of observed values of the dependent variable from its expected value.

Two Lines of Regression

For two variables are \(x\) and \(y\),

• When the dependent variable is \(x\), the two variables are correlated by the line of regression of \(x\) on \(y\).
• When the dependent variable is \(y\), the two variables are correlated by the line of regression of \(y\) on \(x\).

Formula of the Two Lines of Regression

The line of regression of \(x\) on \(y\) is given by the equation \(\left( {x – \bar x} \right) = \frac{{r{\sigma _x}}}{{{\sigma _y}}}\left( {y – \bar y} \right)\)
The line of regression of \(y\) on \(x\) is given by the equation \(\left( {y – \bar y} \right) = \frac{{r{\sigma _y}}}{{{\sigma _x}}}\left( {x – \bar x} \right)\)
Here,
\(\bar x =\) mean of the values of \(x\)
\(\bar y =\) mean of the values of \(y\)
\({\sigma _x} =\) Standard deviation of the values of \(x\) from \(\bar x \)
\({\sigma _y} =\) Standard deviation of the values of \(y\) from \(\bar y \)
\(r =\) Correlation coefficient

PRACTICE EXAM QUESTIONS AT EMBIBE

Alternate Formula of the Line of Regression of x on y

Let \(x\) be the dependent variable and \(y\) be the independent variable. The given data set of variables is \(\left( {{x_1},\,{y_1}} \right),\,\left( {{x_2},\,{y_2}} \right),\,\left( {{x_3},\,{y_3}} \right), \ldots \ldots ,\left( {{x_n},\,{y_n}} \right)\).
We assume that the line of regression of \(x\) on \(y\) is,
\(x = a + by\;\;\;\cdots(1)\)
Here, \(a\) and \(b\) are the constants that help establish a relation between \(x\) and \(y\).
Let \({X_i} = a + b{y_i}\) be the estimated value of \(x = x_i\) for the given value of \(y = y_i\).
Method of least squares involves determining values of \(a\) and \(b\) so that the sum of squares of deviations of the given values of \(X\) from estimated values of \(X\).
\(U = \mathop \sum \limits_{i = 1}^n {\left( {{x_i} – {X_i}} \right)^2}\)
\(\therefore \,U = \mathop \sum \limits_{i = 1}^n {\left[ {{x_i} – \left( {a + b{y_i}} \right)} \right]^2}\)
\(\therefore \,U = \mathop \sum \limits_{i = 1}^n {\left( {{x_i} – a – b{y_i}} \right)^2}\;\;\;\cdots(2)\)
We have to minimise \(U\) and to do that, we take partial derivatives w.r.t. \(a\) and \(b\) and equate it to zero.
\(\frac{{\partial U}}{{\partial a}} = 0\)
\(\therefore \,\frac{\partial }{{\partial a}}\mathop \sum \limits_{i = 1}^n {\left( {{x_i} – a – b{y_i}} \right)^2} = 0\)
\(\therefore \,2\mathop \sum \limits_{i = 1}^n \left( {{x_i} – a – b{y_i}} \right)\left( { – 1} \right) = 0\)
\(\therefore \,\mathop \sum \limits_{i = 1}^n \left( {{x_i} – a – b{y_i}} \right) = 0\)
\(\therefore \,\mathop \sum \limits_{i = 1}^n {x_i} – \mathop \sum \limits_{i = 1}^n a – \mathop \sum \limits_{i = 1}^n b{y_i} = 0\)
\(\therefore \,\mathop \sum \limits_{i = 1}^n {x_i} – na – b\mathop \sum \limits_{i = 1}^n {y_i} = 0\)
\(\therefore \,\mathop \sum \limits_{i = 1}^n {x_i} = na + b\mathop \sum \limits_{i = 1}^n {y_i}\;\;\;\cdots(3)\)
and
\(\frac{{\partial U}}{{\partial b}} = 0\)
\(\therefore \,\frac{\partial }{{\partial b}}\mathop \sum \limits_{i = 1}^n {\left( {{x_i} – a – b{y_i}} \right)^2} = 0\)
\(\therefore \,2\mathop \sum \limits_{i = 1}^n \left( {{x_i} – a – b{y_i}} \right)\left( { – {y_i}} \right) = 0\)
\(\therefore \,\mathop \sum \limits_{i = 1}^n \left( {{x_i}{y_i} – a{y_i} – by_i^2} \right) = 0\)
\(\therefore \,\mathop \sum \limits_{i = 1}^n {x_i}{y_i} – \mathop \sum \limits_{i = 1}^n a{y_i} – \mathop \sum \limits_{i = 1}^n by_i^2 = 0\)
\(\therefore \,\mathop \sum \limits_{i = 1}^n {x_i}{y_i} – a\mathop \sum \limits_{i = 1}^n {y_i} – b\mathop \sum \limits_{i = 1}^n y_i^2 = 0\)
\(\therefore \,\mathop \sum \limits_{i = 1}^n {x_i}{y_i} = a\mathop \sum \limits_{i = 1}^n {y_i} + b\mathop \sum \limits_{i = 1}^n y_i^2 \;\;\;\cdots(4)\)
Dividing equation (3) by \(n\), we get
\(\frac{1}{n}\mathop \sum \limits_{i = 1}^n {x_i} = \frac{1}{n}na + \frac{b}{n}\mathop \sum \limits_{i = 1}^n {y_i}\)
\(\therefore \,\bar x = a + b\bar y\;\;\;\cdots(5)\)
We can see that the line represented by equation (1) satisfies the point (\(\bar x\), \(\bar y\)). Thus, the line of regression of \(x\) on \(y\) passes through the point that shows the mean value of \(x\) and \(y\).
Covariance of variables \(x\) and \(y\) is given by,
\(cov\left( {x,y} \right) = \frac{1}{n}\mathop \sum \limits_{i = 1}^n {x_i}{y_i} – \bar x\bar y\)
\(\therefore \,\frac{1}{n}\mathop \sum \limits_{i = 1}^n {x_i}{y_i} = cov\left( {x,y} \right) + \bar x\bar y\;\;\;\cdots(6)\)
Variance of variable \(y\) is given by,
\(\sigma_y^2 = \frac{1}{n}\mathop \sum \limits{i = 1}^n y_i^2 – {\bar y^2}\)
\(\therefore \,\frac{1}{n}\mathop \sum \limits_{i = 1}^n y_i^2 = \sigma _y^2 + {\bar y^2}\;\;\;\cdots(7)\)
Dividing equation (4) by \(n\), we get
\(\frac{1}{n}\mathop \sum \limits_{i = 1}^n {x_i}{y_i} = \frac{a}{n}\mathop \sum \limits_{i = 1}^n {y_i} + \frac{b}{n}\mathop \sum \limits_{i = 1}^n y_i^2 \;\;\;\cdots(8)\)
We can substitute equations (6), (7) in equation (8) to get,
\(cov\left( {x,y} \right) + \bar x\bar y = a\bar y + b\left( {\sigma _y^2 + {{\bar y}^2}} \right)\;\;\;\cdots(9)\)
Multiplying throughout by \(\bar y\) in equation (5), we get,
\(\bar x\bar y = a\bar y + b{\bar y^2}\;\;\;\cdots(10)\)
We can subtract equations (10) from equation (9) to get,
\(cov\left( {x,y} \right) = b\sigma _y^2\)
\(\therefore \,b = \frac{{cov\left( {x,y} \right)}}{{\sigma _y^2}}\)
We can write equation (1) in a different form as,
\(x = a + by\)
\(y = \frac {x}{b} – a\)
\(\frac {1}{b}\) is the slope of the line of regression of \(x\) on \(y\) and this line passes through the point \((\bar x,\, \bar y)\)
The equation of the line of regression of \(x\) on \(y\) can be written as,
\(\left( {y – \bar y} \right) = \frac{1}{b}\left( {x – \bar x} \right)\)
\(\therefore \,\left( {x – \bar x} \right) = b\left( {y – \bar y} \right)\)
\(\therefore \,\left( {x – \bar x} \right) = \frac{{cov\left( {x,y} \right)}}{{\sigma _y^2}}\left( {y – \bar y} \right)\)
This is the alternate form of the equation of the line of regression of \(x\) on \(y\). We can reach this equation without finding the variables’ correlation coefficient and standard deviation.
Regression coefficient of \(x\) on \(y\), \({b_{xy}} = \frac{{cov\left( {x,y} \right)}}{{\sigma _y^2}} = \frac{{cov\left( {x,y} \right)}}{{var\left( y \right)}}\)
We can also write,
\({b_{xy}} = \frac{{\frac{{\mathop \sum \nolimits_{i = 1}^n ({x_i} – \bar x)({y_i} – \bar y)}}{n}}}{{\frac{{\mathop \sum \nolimits_{i = 1}^n {{({y_i} – \bar y)}^2}}}{n}}}\)
\({b_{xy}} = \frac{{\frac{1}{n}\mathop \sum \nolimits_{i = 1}^n {x_i}{y_i} – \bar x\bar y}}{{\frac{1}{n}\mathop \sum \nolimits_{i = 1}^n y_i^2 – {{\bar y}^2}}}\)
\({b_{xy}} = \frac{{\sum {x_i}{y_i} – n\bar x\bar y}}{{\sum {y_i}^2 – n{{\left( {\bar y} \right)}^2}}}\)

Alternate Formula of the Line of Regression of y on x

If we assume that the line of regression of \(y\) on \(x\) as \(x = p + qy\) and follow the steps above, we can get the alternate form of the equation of the line of regression of \(x\) on \(y\).
\(\therefore \,\left( {y – \bar y} \right) = \frac{{cov\left( {x,y} \right)}}{{\sigma _x^2}}\left( {x – \bar x} \right)\)
Regression coefficient of \(y\) on \(x\), \({b_{yx}} = \frac{{cov\left( {x,y} \right)}}{{\sigma _x^2}} = \frac{{cov\left( {x,y} \right)}}{{var\left( x \right)}}\)
We can also write,
\({b_{yx}} = \frac{{\frac{{\mathop \sum \nolimits_{i = 1}^n ({x_i} – \bar x)({y_i} – \bar y)}}{n}}}{{\frac{{\mathop \sum \nolimits_{i = 1}^n {{({x_i} – \bar x)}^2}}}{n}}}\)
\({b_{yx}} = \frac{{\frac{1}{n}\mathop \sum \nolimits_{i = 1}^n {x_i}{y_i} – \bar x\bar y}}{{\frac{1}{n}\mathop \sum \nolimits_{i = 1}^n x_i^2 – {{\bar x}^2}}}\)
\({b_{yx}} = \frac{{\sum {x_i}{y_i} – n\bar x\bar y}}{{\sum {x_i}^2 – n{{\left( {\bar x} \right)}^2}}}\)

Alternative Regression Methods

Linear regression may not yield accurate results when the dependent variable does not have constant variance or if the dependent variables have many outliers (points that are very far from the mean). In such cases, using an alternative method of regression may help. We can try minimising the absolute value of the deviations of observed values of the dependent variable from its expected value. We can also try removing the outliers from the analysis to help get better results.
There are more advanced methods of regression analysis that are not included here because of their complexity.
A few of them are,

• Polynomial Regression
• Logistic Regression
• Quantile Regression
• Ridge Regression
• Lasso Regression

Solved Examples – Alternate Form of Regression Equation

Below are a few solved examples that can help in getting a better idea.

Q.1. \(20\) pairs of observations give the following results,
\(\bar x = 180,\,\bar y = 80,\,\sum {\left( {{x_i} – \bar x} \right)^2} = 1000,\,\sum {\left( {{y_i} – \bar y} \right)^2} = 300,\,\sum \left( {{x_i} – \bar x} \right)\left( {{y_i} – \bar y} \right) = – 250\)
Find the line of regression of \(y\) on \(x\).
Ans:
These results are for \(20\) pairs of observations.
\(\therefore n = 20\)
\({b_{yx}} = \frac{{\frac{{\mathop \sum \nolimits_{i = 1}^n ({x_i} – \bar x)({y_i} – \bar y)}}{n}}}{{\frac{{\mathop \sum \nolimits_{i = 1}^n {{({x_i} – \bar x)}^2}}}{n}}}\)
\(\therefore \,{b_{yx}} = \frac{{\frac{{ – 250}}{{20}}}}{{\frac{{1200}}{{20}}}} = \frac{{ – 250}}{{1000}} = – 0.25\)
The linear regression equation of \(y\) on \(x\),
\(\left( {y – \bar y} \right) = {b_{yx}}\left( {x – \bar x} \right)\)
\(\therefore \,\left( {y – 80} \right) = – 0.25\left( {x – 180} \right)\)
\(\therefore \,y – 80 = – 0.25x + 45\)
\(\therefore \,y = – 0.25x + 45 + 80\)
\(\therefore \,y = – 0.25x + 125\)

Q.2. Find the linear regression equation of \(y\) on \(x\) for the following data set below.

Ans:
To find the linear regression equation of \(y\) on \(x\) we require \(\bar x,\,\bar y\) and \(b_{yx}\).
For finding \(b_{yx}\), we require \(\sum {x_i}{y_i}\), \(\sum {x_i}^2\), \(\bar x\) and \(\bar y\).

\(\bar x = \frac{{\sum {x_i}}}{n} = \frac{{20}}{4} = 5\)
\(\bar y = \frac{{\sum {y_i}}}{n} = \frac{{25}}{4} = 6.25\)
\({b_{yx}} = \frac{{\sum {x_i}{y_i} – n\bar x\bar y}}{{\sum {x_i}^2 – n{{\left( {\bar x} \right)}^2}}}\)
\(\therefore \,{b_{yx}} = \frac{{144 – \left( {4 \times 5 \times 6.25} \right)}}{{120 – \left( {4 \times {5^2}} \right)}} = \frac{{144 – 125}}{{120 – 100}} = \frac{{19}}{{20}} = 0.95\)
The linear regression equation of \(y\) on \(x\),
\(\left( {y – \bar y} \right) = {b_{yx}}\left( {x – \bar x} \right)\)
\(\therefore \,\left( {y – 6.25} \right) = 0.95\left( {x – 5} \right)\)
\(\therefore \,y – 6.25 = 0.95x – 4.75\)
\(\therefore \,y = 0.95x – 4.75 + 6.25\)
\(\therefore \,y = 1.5 + 0.95x\)

Q.3. For a bivariate data set: \(\sum x = 15,\,\sum y = 15,\,\sum {y^2} = 49,\,\sum xy = 44\) and \(n = 5\). Find \(x\), when \(y = 9\).
Ans:
We have to find \(x\) for a given value of \(y\), so we have to find the linear regression equation of \(x\) on \(y\).
To find the linear regression equation of \(x\) on \(y\) we require \(\bar x,\,\bar y\) and \(b_{xy}\).
For finding \(b_{xy}\), we require \(\sum {x_i}{y_i},\,\sum y_i^2,\,\bar x\) and \(\bar y\).
\(\bar x = \frac{{\sum {x_i}}}{n} = \frac{{15}}{5} = 3\)
\(\bar y = \frac{{\sum {y_i}}}{n} = \frac{{15}}{5} = 3\)
\({b_{xy}} = \frac{{\sum {x_i}{y_i} – n\bar x\bar y}}{{\sum {y_i}^2 – n{{\left( {\bar y} \right)}^2}}}\)
\(\therefore \,{b_{xy}} = \frac{{44 – \left( {5 \times 3 \times 3} \right)}}{{49 – \left( {5 \times {3^2}} \right)}}\)
\(\therefore \,{b_{xy}} = \frac{{44 – 45}}{{49 – 45}}\)
\(\therefore \,{b_{xy}} = \frac{{ – 1}}{4} = – 0.25\)
The linear regression equation of \(x\) on \(y\),
\(\left( {x – \bar x} \right) = {b_{xy}}\left( {y – \bar y} \right)\)
\(\therefore \,\left( {x – 3} \right) = – 0.25\left( {y – 3} \right)\)
\(\therefore \,x – 3 = – 0.25y + 0.75\)
\(\therefore \,x = – 0.25y + 0.75 + 3\)
\(\therefore \,x = – 0.25y + 3.75\)
When, \(y = 7\),
\(x = – 0.25 \times 9 + 3.75\)
\(\therefore \,x = – 2.25 + 3.75\)
\(\therefore \,x = 1.50\)

Q.4. Find the correct regression equation for the following dataset:

Ans:
Since \(x\) is the independent variable and \(y\) is the dependent variable here, we have to find the linear regression equation of \(y\) on \(x\).
To find the linear regression equation of \(y\) on \(x\) we require \(\bar x,\,\bar y\) and \(b_{yx}\).
For finding \(b_{yx}\), we require \(\sum {x_i}{y_i},\,\sum x_i^2,\,\bar x\) and \(\bar y\).

\(\bar x = \frac{{\sum {x_i}}}{n} = \frac{{36}}{6} = 6\)
\(\bar y = \frac{{\sum {y_i}}}{n} = \frac{{60}}{6} = 10\)
\({b_{yx}} = \frac{{\sum {x_i}{y_i} – n\bar x\bar y}}{{\sum {x_i}^2 – n{{\left( {\bar x} \right)}^2}}}\)
\(\therefore \,{b_{yx}} = \frac{{293 – \left( {6 \times 6 \times 10} \right)}}{{266 – \left( {6 \times {6^2}} \right)}} = \frac{{293 – 360}}{{266 – 216}} = \frac{{ – 67}}{{50}} = – 1.34\)
The linear regression equation of \(y\) on \(x\),
\(\left( {y – \bar y} \right) = {b_{yx}}\left( {x – \bar x} \right)\)
\(\therefore \,\left( {y – 10} \right) = – 1.34\left( {x – 6} \right)\)
\(\therefore \,y – 10 = – 1.34x + 8.04\)
\(\therefore \,y = – 1.34x + 8.04 + 10\)
\(\therefore \,y = – 1.34x + 18.04\)

Q.5. A survey conducted to study the feasibility of organic farming gave the following data on organic fertiliser requirements and grain yield.

Historical data shows that \(90\,{\rm{kg}}\) of a chemical fertiliser produces a grain yield of \(70\,{\rm{kg}}\). If switching to organic farming can initially cause a drop in output by \(10\%\), determine if it is statistically correct to make this switch.
Ans:
Let the fertiliser requirement be \(x\) and the grain yield by \(y\).
We have to find \(y\) for \(x = 90\), so we require a linear regression equation of \(y\) on \(x\).
To find the linear regression equation of \(y\) on \(x\) we require \(\bar x,\,\bar y\) and \(b_{yx}\).
Ans:
Since \(x\) is the independent variable and \(y\) is the dependent variable here, we have to find the linear regression equation of \(y\) on \(x\).
To find the linear regression equation of \(y\) on \(x\) we require \(\bar x,\,\bar y\) and \(b_{yx}\).
For finding \(b_{yx}\), we require \(\sum {x_i}{y_i},\,\sum x_i^2,\,\bar x\) and \(\bar y\).

\(\bar x = \frac{{\sum {x_i}}}{n} = \frac{{330}}{6} = 55\)
\(\bar y = \frac{{\sum {y_i}}}{n} = \frac{{314}}{6} = \frac{{157}}{3}\)
\({b_{yx}} = \frac{{\sum {x_i}{y_i} – n\bar x\bar y}}{{\sum {x_i}^2 – n{{\left( {\bar x} \right)}^2}}}\)
\(\therefore \,{b_{yx}} = \frac{{17930 – \left( {6 \times 55 \times \frac{{157}}{3}} \right)}}{{19900 – \left( {6 \times {{55}^2}} \right)}} = \frac{{17930 – 17270}}{{19900 – 18150}} = \frac{{660}}{{1750}} = \frac{{66}}{{175}}\)
The linear regression equation of \(y\) on \(x\),
\(\left( {y – \bar y} \right) = {b_{yx}}\left( {x – \bar x} \right)\)
\(\therefore \,\left( {y – \frac{{157}}{3}} \right) = \frac{{66}}{{175}}\left( {x – 55} \right)\)
When \(x = 90\),
\(y – \frac{{157}}{3} = \frac{{66}}{{175}}\left( {90 – 55} \right)\)
\(\therefore \,y = 13.2 + \frac{{157}}{3}\)
\(\therefore \,y = \frac{{196.6}}{3} = 65.53\)
Thus, \(90\, {\rm{kg}}\) of organic fertiliser produces an output of \(65.53\, {\rm{kg}}\) of grain.
The same amount of chemical fertiliser produces a \(70\,{\rm{kg}}\) grain yield. Considering the initial drop in yield of \(10\%\) when switching to organic farming, acceptable yield is,
\(70 – \left( {10\% \;{\rm{of}}\;70} \right) = 70 – \left( {\frac{{10}}{{100}} \times 70} \right) = 63\)
Since organic fertilisers produce a better yield than the acceptable limit, it is correct to switch to organic farming from a statistical point of view.

ATTEMPT MOCK TESTS ON EMBIBE

Summary

Broadly, types of regression are linear regression, non-linear regression, bivariate regression, and multivariate regression. Lines of regression can be found by graphical method or by an analytical approach using the method of least squares. An alternate form of the regression equation involves representing the regression coefficients in terms of covariance and variance of the variables.

In datasets where certain points lie significantly far from the mean, linear regression may give incorrect results. Using alternative methods like minimising absolute deviation and removing the outliers can help. Other methods of regression include polynomial regression, logistic regression, quantile regression, ridge regression, and lasso regression.

Frequently Asked Questions (FAQs)

Students might be having many questions with respect to the Alternate Form of Regression Equation. Here are a few commonly asked questions and answers.

Q.1. What is an alternative to regression?
Ans: An alternative to regression can be a relationship between variables based on scientific principles and empirical relations and not entirely dependent on historical data.

Q.2. What are the other forms of regression?
Ans: Other forms of regressions include non-linear model, weighted least squares regression, polynomial regressions, logistic regression, etc.

Q.3. How do you write a regression equation?
Ans: Linear regression equation of \(x\) on \(y\), \(\left( {x – \bar x} \right) = \frac{{cov\left( {x,y} \right)}}{{\sigma _y^2}}\left( {y – \bar y} \right)\)
Linear regression equation of \(y\) on \(x\), \(\left( {y – \bar y} \right) = \frac{{cov\left( {x,y} \right)}}{{\sigma _x^2}}\left( {x – \bar x} \right)\)
Here,
\(\bar x,\, \bar y =\) mean of the values of \(x\) and \(y\), respectively
\({\sigma _x},\,{\sigma _y} = \) Standard deviations of the values of \(x\) from \(\bar x\) and the values of \(y\) from \(\bar y\) respectively
\(cov\left( {x,y} \right) = \) covariance of \(x\) and \(y\)

Q.4. What is another name for regression equation?
Ans: Another name for regression equation is estimation equation.

Q.5. What is the relation between correlation and regression?
Ans: Correlation measures how strongly two variables are related to each other. Higher correlation implies the estimation of dependent variables, using the regression equation is more accurate.

We hope this information about the Alternate Form of Regression Equation has been helpful. If you have any doubts, comment in the section below, and we will get back to you.