How To Tell If Data Is Linear

I like a lot the ANOVA-based answer by Stephan Kolassa. However, I would also similar to offer a slightly unlike perspective.

Showtime of all, consider the reason why you're testing for nonlinearity. If you desire to test the assumptions of ordinary least squares to estimate the elementary linear regression model, note that if you so want to employ the estimated model to perform other tests (east.one thousand., test whether the correlation between $10$ and $Y$ is statistically pregnant), the resulting test will be a blended test, whose Type I and Type II error rates won't be the nominal ones. This is i of multiple reasons why, instead than formally testing the assumptions of linear regression, you may want to apply plots in lodge to sympathize if those assumptions are reasonable. Another reason is that the more tests you lot perform, the more likely you are to go a significant examination result even if the null is true (after all, linearity of the human relationship between $X$ and $Y$ is non the only supposition of the simple linear regression model), and closely related to this reason in that location'due south the fact that assumption tests have themselves assumptions!

For example, post-obit Stephan Kolassa's example, permit's build a uncomplicated regression model:

          set.seed(i) 20 <- runif(100) yy <- xx^2+rnorm(100,0,0.i) plot(xx,yy) linear.model <- lm(yy ~ xx)                  

The plot part for linear models shows a host of plots whose goal is exactly to requite y'all an idea about the validity of the assumptions backside the linear model and the OLS interpretation method. The purpose of the first of these plots, the residuals vs fitted plot, is exactly to show if there are deviations from the assumption of a linear relationship between the predictor $X$ and the response $Y$:

          plot(linear.model)                  

You lot can conspicuously see that there is a quadratic trend between fitted values and residuals, thus the supposition that $Y$ is a linear function of $X$ is questionable.

If, however, you are determined on using a statistical test to verify the assumption of linearity, so you're faced with the result that, as noted past Stephan Kolassa, there are infinitely many possible forms of nonlinearity, and then you cannot possibly devise a single examination for all of them. You need to make up one's mind your alternatives and so you lot can test for them. Now, if all your alternatives are polynomials, then y'all don't even need ANOVA, because past default R computes orthogonal polynomials. Let's examination 4 alternatives, i.east., a linear polynomial, a quadratic one, a cubic i and a quartic one. Of course, looking at the residual vs fitted plot, in that location's not evidence for an college than degree 2 model here. Still, nosotros include the higher caste models to show how to operate in a more full general case. Nosotros just need ane fit to compare all four models:

          quartic.model <- lm(yy ~ poly(xx,4)) summary(quartic.model) Call: lm(formula = yy ~ poly(xx, 4))  Residuals:       Min        1Q    Median        3Q       Max  -0.175678 -0.061429 -0.007403  0.056324  0.264612   Coefficients:              Guess Std. Mistake t value Pr(>|t|)     (Intercept)   0.33729    0.00947  35.617  < 2e-sixteen *** poly(xx, 4)one  two.78089    0.09470  29.365  < 2e-16 *** poly(twenty, four)2  0.64132    0.09470   6.772 1.05e-09 *** poly(xx, 4)3  0.04490    0.09470   0.474    0.636     poly(20, 4)4  0.11722    0.09470   1.238    0.219                  

As yous can see, the p-values for the beginning and 2nd caste term are extremely low, meaning that a linear fit is bereft, but the p-values for the third and quaternary term are much larger, meaning that third or college caste models are not justified. Thus, nosotros select the 2d degree model. Notation that this is only valid considering R is fitting orthogonal polynomials (don't endeavour to do this when fitting raw polynomials!). The result would have been the same if we had used ANOVA. Every bit a matter of fact, the squares of the t-statistics here are equal to the F-statistics of the ANOVA test:

          linear.model <- lm(yy ~ poly(twenty,1)) quadratic.model <- lm(yy ~ poly(xx,2)) cubic.model <- lm(yy ~ poly(20,3)) anova(linear.model, quadratic.model, cubic.model, quartic.model) Analysis of Variance Table  Model 1: yy ~ poly(xx, 1) Model 2: yy ~ poly(twenty, two) Model 3: yy ~ poly(20, 3) Model iv: yy ~ poly(twenty, iv)   Res.Df     RSS Df Sum of Sq       F    Pr(>F)     1     98 1.27901                                    2     97 0.86772  i   0.41129 45.8622 1.049e-09 *** 3     96 0.86570  ane   0.00202  0.2248    0.6365     4     95 0.85196  1   0.01374  1.5322    0.2188                  

For case, 6.772^2 = 45.85998, which is not exactly 45.8622 but pretty close, taking into business relationship numerical errors.

The advantage of the ANOVA examination comes into play when you lot want to explore not-polynomial models, as long every bit they're all nested. Two or more models $M_1,\dots,M_N$ are nested if the predictors of $M_i$ are a subset of the predictors of $M_{i+one}$, for each $i$. For example, let's consider a cubic spline model with i interior knot placed at the median of xx. The cubic spline basis includes linear, 2d and 3rd caste polynomials, thus the linear.model, the quadratic.model and the cubic.model are all nested models of the following spline.model:

          spline.model <- lm(yy ~ bs(xx,knots = quantile(xx,prob=0.five)))                  

The quartic.model is not a nested model of the spline.model (nor is the vice versa true), then we must leave it out of our ANOVA test:

          anova(linear.model, quadratic.model,cubic.model,spline.model) Analysis of Variance Tabular array  Model 1: yy ~ poly(xx, one) Model two: yy ~ poly(xx, 2) Model three: yy ~ poly(twenty, 3) Model 4: yy ~ bs(xx, knots = quantile(xx, prob = 0.five))   Res.Df     RSS Df Sum of Sq       F    Pr(>F)     one     98 one.27901                                    2     97 0.86772  one   0.41129 46.1651 9.455e-10 *** three     96 0.86570  1   0.00202  0.2263    0.6354     iv     95 0.84637  1   0.01933  2.1699    0.1440                  

Over again, we run across that a quadratic fit is justified, only we take no reason to refuse the hypothesis of a quadratic model, in favour of a cubic or a spline fit culling.

Finally, if you would similar to exam also not-nested model (for example, you would like to exam a linear model, a spline model and a nonlinear model such as a Gaussian Procedure), and so I don't call back there are hypothesis tests for that. In this case your best bet is cantankerous-validation.

How To Tell If Data Is Linear,

Source: https://stats.stackexchange.com/questions/239141/statistical-test-to-determine-if-a-relationship-is-linear

Posted by: wagnergear1974.blogspot.com

Share this post