Talk:Total least squares

From Wikipedia, the free encyclopedia

Can you clarify: Are the noises in A iid and the noises in b iid (but with possibly a differenct common distribution from the noises in A)-- OR, do the noises in A have the same distribution as the noises in b?

To my best knowledge, the connection between the residuals and the real noise distribution is still not clear in any literature. Only thing we can say is that the residuals have even probability in positive and negative values. Based on that, noises in A and B have even distribution functions. If the real noise has odd pdf such as chi-distribution, then TLS may not work well. It is because TLS allows the negative correction in elements of A and b. Rather than speaking distribution, we can say that TLS minimizes the estimate of noise powers equally in A and b. I am sorry but I can describe only a very rough picture. S. Jo 01:40, 18 April 2006 (UTC)

Contents

[edit] This page is wrong

This page is wrong. Error-in-variables refers to a model where the independent variables are measured with error. There are lots of possible solutions, of which total least squares is only one possibility (and one that's far from universally accepted. Economists would never use it, for example.) -- Walt Pohl 02:44, 27 November 2006 (UTC)

I agree (that it's incomplete). In the eiv literature method of moments estimators (plural!) are more in use. Eg for the example on this page, see Amemiya et al 1987. There are also several other estimators: SIMEX, estimating equations with adjusted score functions,...more? Libraries have been written on this, and not all on "total least squares"!
Carroll et al. heaviliy criticise orthogonal regression. In this specific example their criticism does not apply because the "\approx" sign I presume means that the relationship is near perfect. As soon as that is not the case however, orthogonal regression goes wrong. Apparently this problem is not encountered much in computer science (??) but in other disciplines (econometrics, epidemiology, psychology, medicine, agricultural sciences, etc. etc.) it is more or less always the case that relationships to be estimated are imperfect.
Besides that it is difficult to understand, this explanation seems very tailored to the treatment of this problem within one specific field.
87.219.191.214 11:53, 5 April 2007 (UTC)

[edit] Total Least Squares, Orthogonal, Errors-in-variables

I think we need some clarification on the terminology used throughout the literature. I am not a real statistician, but to me Orthogonal regression as explained <A HREF="http://www.nlreg.com/orthogonal.htm"> here </A> is the same as TLS, at least it minimizes the same distances... could anyone elaborate? I think it would really add to the usability of the method. Jeroenemans 16:53, 5 December 2006 (UTC)

In my view, "Orthogonal" and TLS are similar, but "Errors-in-variables" still leaves freedom in which direction to measure error. It all depends to what degree you would like to have the dependant variable contribute to error. So, in the case of the basic 2-dimensional model you are refering to, one should not necessarily assume orthogonal distance meausurement as optimal solution. OK?Witger 18:03, 5 December 2006 (UTC)

[edit] Page revised

This page has been revised to bring it in line with other articles on least squares. As it stands, it is mostly relevant to the physical sciences, where there is experimental error on all measured quantities. It is obvious from the discussion above that in other fields other methods are more likely to be used, but I have no experience of them. The earlier version mentions Data Least Squares, and Structured Total Least Squares, but gave no details, so they have been omitted, for now.

Regarding the comment "this page is wrong", I would say rather that it's the title that is wrong. It should always have been Total least squares as that is what it was about. Maybe Errors-in-variables is another topic altogether? Petergans (talk) 15:34, 16 February 2008 (UTC)

I think that's right. (Total least squares is a technique for error-in-variables that makes sense for the physical sciences (where you can plausibly know that two different variables have the same size of measurement error, but it makes less sense in the social sciences, where instrumental variables would be preferred. -- Walt Pohl (talk) 20:21, 16 February 2008 (UTC)
Error size has no meaning in physical measurements, unless it's of the same thing and in the same units. Only relative sizes matter. Is this what you mean? Pgpotvin (talk) 05:34, 22 February 2008 (UTC)

[edit] Should we move to his page to Total Least Squares?

I was thinking (inspired by Peter Gans) that we should just move this page to total least squares, and start a new page on error-in-variables. Does anyone have any objections? -- Walt Pohl (talk) 20:34, 16 February 2008 (UTC)

I would support that suggestion. I've just finished revising regression analysis where there is a link in Underlying assumptions to this article. That link would be better directed to the new page. Petergans (talk) 14:04, 20 February 2008 (UTC)
Since no one else seems to have an opinion, I went ahead and did it. -- Walt Pohl (talk) 07:20, 21 February 2008 (UTC)

[edit] Definitions, please

Peter, please define your symbols. 'delta y', 'delta beta', 'K' and 'F', for instance. And it would be nice to see how the 'F' equations are condition statements (constraints). And what is meant by condition? Simultaneity? I find this article difficult to read and I can't imagine what a novice could make of it. (P.S. I agree with Walt Pohl on the absence of consensus on TLS. I am personally suspicious of it.) Pgpotvin (talk) 05:30, 22 February 2008 (UTC)

This is inherently a very technical subject, not suitable for novices. "Some algebraic manipulation" is long and complicated, so I opted for a brief summary. As to whether TLS is innocent or not, that's for the jury to decide. Petergans (talk) 08:02, 22 February 2008 (UTC)