Linear Regression

[sipanivishal]

Hi David,

Gujrati talked about linearity of parameter.To me that is insane because b^2 is a constant and it can replaced by another constant,say c. Then it would become linear with respect to c and non linear with b^2. That is just not right.
What is your thought on it.

Thanks
Sipani

 

[David Harper CFA FRM]

Hi Sipani,

I can relate to this, is my first thought! Partly, it is semantic: he is only saying, this is the meaning of linear, for our purposes, where our purposes happen to be an ordinary least square (OLS) regression. He is echoing that linear params are an assumption (a precondition) to the OLS that is reviewed. It's true, you can look at y = m^2(x) + b and translate into y = nx + b. But that is "after the fact."

"Before the fact" you don't know m (slope) or the partial slope (in the case of multiple regression). You are estimating the param(s) by drawing thru the data a least squares line. The rules of this OLS method (as they are based on partial derivatives with respect to these params) happen to require B is not raised to power (so, it may be easier to sort this out with the exception that looks like y = b1^b2(x1) + b2^b1(x2) as this is also nonlinear params. As irony would have it, in the OLS procedure, the truth of this is due to the fact the params are actually the variables to be solved for and in the linear system they need to be linear.). Put another way, the OLS treats the params as linear on the way to finding the least square line. I, too, find this confusing because visually it is counterintuitive. But I hope this helps: 1. it's not really about the definition of "linear" which is ambigous on its own and Gujarati is himself saying by offering 2 definitions, 2. before the OLS, you don't know the params, you are estimating them with samples (i.e., before the fact, the params are variables!), 3. it's just a precondition for using OLS based on the OLS procedure.

David

 

[David Harper CFA FRM]

Hi Sipani,

I was just looking at the Gujarati text to see if he gives any more explain on this (non linear params). While i don't see it directly, Appendix 6a does indirectly show it. In A 6-A shows how the formulas are derived that estimate the parameters (b1, b2). These formulas, you can see in A 6-A are based on differentiating the residual sum of squares (RSS). See how 6.a.2 and 6.a.3 are "with respect to the params b1 and b2"? IMO, this is the answer to your question: the OLS gives estimates for the params based on this differentiation - these are only relatively simple differentials b/c the params b1 and b2 are first order\

...while you are right that after the fact b^2 = c, "before the fact," these are variables...

hope that helps. David