Generalized additive models with integrated smoothness estimation
Description
Fits a generalized additive model (GAM) to
data, the term ‘GAM’ being taken to include any quadratically penalized GLM and a variety of
other models estimated by a quadratically penalised likelihood type approach (see family.mgcv).
The degree of smoothness of model terms is estimated as part of
fitting. gam can also fit any GLM subject to multiple quadratic penalties (including
estimation of degree of penalization). Confidence/credible intervals are readily
available for any quantity predicted using a fitted model.
Smooth terms are represented using penalized regression splines (or similar smoothers)
with smoothing parameters selected by GCV/UBRE/AIC/REML/NCV or by regression splines with
fixed degrees of freedom (mixtures of the two are permitted). Multi-dimensional smooths are
available using penalized thin plate regression splines (isotropic) or tensor product splines
(when an isotropic smooth is inappropriate), and users can add smooths.
Linear functionals of smooths can also be included in models.
For an overview of the smooths available see smooth.terms.
For more on specifying models see gam.models, random.effects and linear.functional.terms. For more on model selection see gam.selection. Do read gam.check and choose.k.
See package gam, for GAMs via the original Hastie and Tibshirani approach (see details for differences to this implementation).
For very large datasets see bam, for mixed GAM see gamm and random.effects.
Usage
gam(formula,family=gaussian(),data=list(),weights=NULL,subset=NULL,
na.action,offset=NULL,method="GCV.Cp",
optimizer=c("outer","newton"),control=list(),scale=0,
select=FALSE,knots=NULL,sp=NULL,min.sp=NULL,H=NULL,gamma=1,
fit=TRUE,paraPen=NULL,G=NULL,in.out,drop.unused.levels=TRUE,
drop.intercept=NULL,nei=NULL,discrete=FALSE,...)
Arguments
formula |
A GAM formula, or a list of formulae (see formula.gam and also gam.models).
These are exactly like the formula for a GLM except that smooth terms, s, te, ti
and t2, can be added to the right hand side to specify that the linear predictor depends on smooth functions of predictors (or linear functionals of these).
|
family |
This is a family object specifying the distribution and link to use in
fitting etc (see glm and family). See
family.mgcv for a full list of what is available, which goes well beyond exponential family.
Note that quasi families actually result in the use of extended quasi-likelihood
if method is set to a RE/ML method (McCullagh and Nelder, 1989, 9.6).
|
data |
A data frame or list containing the model response variable and
covariates required by the formula. By default the variables are taken
from environment(formula): typically the environment from
which gam is called.
|
weights |
prior weights on the contribution of the data to the log likelihood. Note that a weight of 2, for example,
is equivalent to having made exactly the same observation twice. If you want to re-weight the contributions
of each datum without changing the overall magnitude of the log likelihood, then you should normalize the weights
(e.g. weights <- weights/mean(weights)).
|
subset |
an optional vector specifying a subset of observations to be
used in the fitting process.
|
na.action |
a function which indicates what should happen when the data
contain ‘NA’s. The default is set by the ‘na.action’ setting
of ‘options’, and is ‘na.fail’ if that is unset. The
“factory-fresh” default is ‘na.omit’.
|
offset |
Can be used to supply a model offset for use in fitting. Note
that this offset will always be completely ignored when predicting, unlike an offset
included in formula (this used to conform to the behaviour of
lm and glm).
|
control |
A list of fit control parameters to replace defaults returned by
gam.control. Values not set assume default values.
|
method |
The smoothing parameter estimation method. "GCV.Cp" to use GCV for unknown scale parameter and
Mallows' Cp/UBRE/AIC for known scale. "GACV.Cp" is equivalent, but using GACV in place of GCV. "NCV"
for neighbourhood cross-validation using the neighbourhood structure speficied by nei ("QNCV" for numerically more ribust version). "REML"
for REML estimation, including of unknown scale, "P-REML" for REML estimation, but using a Pearson estimate
of the scale. "ML" and "P-ML" are similar, but using maximum likelihood in place of REML. Beyond the
exponential family "REML" is the default, and the only other options are "ML", "NCV" or "QNCV".
|
optimizer |
An array specifying the numerical optimization method to use to optimize the smoothing
parameter estimation criterion (given by method). "outer"
for the direct nested optimization approach. "outer" can use several alternative optimizers, specified in the
second element of optimizer: "newton" (default), "bfgs", "optim" or "nlm". "efs"
for the extended Fellner Schall method of Wood and Fasiolo (2017).
|
scale |
If this is positive then it is taken as the known scale parameter. Negative signals that the
scale parameter is unknown. 0 signals that the scale parameter is 1 for Poisson and binomial and unknown otherwise.
Note that (RE)ML methods can only work with scale parameter 1 for the Poisson and binomial cases.
|
select |
If this is TRUE then gam can add an extra penalty to each term so
that it can be penalized to zero. This means that the smoothing parameter estimation that is
part of fitting can completely remove terms from the model. If the corresponding
smoothing parameter is estimated as zero then the extra penalty has no effect. Use gamma to increase level of penalization.
|
knots |
this is an optional list containing user specified knot values to be used for basis construction.
For most bases the user simply supplies the knots to be used, which must match up with the k value
supplied (note that the number of knots is not always just k).
See tprs for what happens in the "tp"/"ts" case.
Different terms can use different numbers of knots, unless they share a covariate.
|
sp |
A vector of smoothing parameters can be provided here.
Smoothing parameters must be supplied in the order that the smooth terms appear in the model
formula. Negative elements indicate that the parameter should be estimated, and hence a mixture
of fixed and estimated parameters is possible. If smooths share smoothing parameters then length(sp)
must correspond to the number of underlying smoothing parameters.
|
min.sp |
Lower bounds can be supplied for the smoothing parameters. Note
that if this option is used then the smoothing parameters full.sp, in the
returned object, will need to be added to what is supplied here to get the
smoothing parameters actually multiplying the penalties. length(min.sp) should
always be the same as the total number of penalties (so it may be longer than sp,
if smooths share smoothing parameters).
|
H |
A user supplied fixed quadratic penalty on the parameters of the
GAM can be supplied, with this as its coefficient matrix. A common use of this term is
to add a ridge penalty to the parameters of the GAM in circumstances in which the model
is close to un-identifiable on the scale of the linear predictor, but perfectly well
defined on the response scale.
|
gamma |
Increase this beyond 1 to produce smoother models. gamma multiplies the effective degrees of freedom in the GCV or UBRE/AIC. n/gamma can be viewed as an effective sample size in the GCV score, and this also enables it to be used with REML/ML. Ignored with P-RE/ML or the efs optimizer.
|
fit |
If this argument is TRUE then gam sets up the model and fits it, but if it is
FALSE then the model is set up and an object G containing what
would be required to fit is returned is returned. See argument G.
|
paraPen |
optional list specifying any penalties to be applied to parametric model terms.
gam.models explains more.
|
G |
Usually NULL, but may contain the object returned by a previous call to gam with
fit=FALSE, in which case all other arguments are ignored except for
sp, gamma, in.out, scale, control, method optimizer and fit.
|
in.out |
optional list for initializing outer iteration. If supplied
then this must contain two elements: sp should be an array of
initialization values for all smoothing parameters (there must be a value for
all smoothing parameters, whether fixed or to be estimated, but those for
fixed s.p.s are not used); scale is the typical scale of the GCV/UBRE function,
for passing to the outer optimizer, or the the initial value of the scale parameter, if this is to
be estimated by RE/ML.
|
drop.unused.levels |
by default unused levels are dropped from factors before fitting. For some smooths
involving factor variables you might want to turn this off. Only do so if you know what you are doing.
|
drop.intercept |
Set to TRUE to force the model to really not have a constant in the parametric model part,
even with factor variables present. Can be vector when formula is a list.
|
nei |
A list specifying the neighbourhood structure for NCV. k is the vector of indices to be dropped for each neighbourhood and m gives the end of each neighbourhood. So nei$k[(nei$m[j-1]+1):nei$m[j]] gives the points dropped for the neighbourhood j. i is the vector of indices of points to predict, with corresponding endpoints mi. So nei$i[(nei$mi[j-1]+1):nei$mi[j]] indexes the points to predict for neighbourhood j. If nei==NULL (or k or m are missing) then leave-one-out cross validation is obtained. If jackknife is supplied then TRUE indicates to use raw jackknife covariances estimator and FALSE to use the conventional Bayes estimate. If not supplied then the estimator accounting for neighbourhood structure is used. jackknife ignored when method is not NCV.
|
discrete |
experimental option for setting up models for use with discrete methods employed in bam. Do not modify.
|
... |
further arguments for
passing on e.g. to gam.fit (such as mustart).
|
Details
A generalized additive model (GAM) is a generalized linear model (GLM) in which the linear
predictor is given by a user specified sum of smooth functions of the covariates plus a
conventional parametric component of the linear predictor. A simple example is:
log{E(yi)}=α+f1(x1i)+f2(x2i)
where the (independent) response variables yi∼Poi, and
f1 and f2 are smooth functions of covariates x1 and
x2. The log is an example of a link function. Note that to be identifiable the model
requires constraints on the smooth functions. By default these are imposed automatically and require that the function sums to zero over the observed covariate values (the presence of a metric by variable is the only case which usually suppresses this).
If absolutely any smooth functions were allowed in model fitting then maximum likelihood
estimation of such models would invariably result in complex over-fitting estimates of
f1 and f2. For this reason the models are usually fit by
penalized likelihood
maximization, in which the model (negative log) likelihood is modified by the addition of
a penalty for each smooth function, penalizing its ‘wiggliness’. To control the trade-off
between penalizing wiggliness and penalizing badness of fit each penalty is multiplied by
an associated smoothing parameter: how to estimate these parameters, and
how to practically represent the smooth functions are the main statistical questions
introduced by moving from GLMs to GAMs.
The mgcv implementation of gam represents the smooth functions using
penalized regression splines, and by default uses basis functions for these splines that
are designed to be optimal, given the number basis functions used. The smooth terms can be
functions of any number of covariates and the user has some control over how smoothness of
the functions is measured.
gam in mgcv solves the smoothing parameter estimation problem by using the
Generalized Cross Validation (GCV) criterion
nD/(n−DoF)2
or an Un-Biased Risk Estimator (UBRE )criterion
D/n+2sDoF/n−s
where D is the deviance, n the number of data, s
the scale parameter and
DoF the effective degrees of freedom of the model. Notice that UBRE is effectively
just AIC rescaled, but is only used when s is known.
Alternatives are GACV, NCV or a Laplace approximation to REML. There
is some evidence that the latter may actually be the most effective choice.
The main computational challenge solved by the mgcv package is to optimize the smoothness selection criteria efficiently and reliably.
Broadly gam works by first constructing basis functions and one or more quadratic penalty
coefficient matrices for each smooth term in the model formula, obtaining a model matrix for
the strictly parametric part of the model formula, and combining these to obtain a
complete model matrix (/design matrix) and a set of penalty matrices for the smooth terms.
The linear identifiability constraints are also obtained at this point. The model is
fit using gam.fit, gam.fit3 or variants, which are modifications
of glm.fit. The GAM
penalized likelihood maximization problem is solved by Penalized Iteratively
Re-weighted Least Squares (P-IRLS) (see e.g. Wood 2000).
Smoothing parameter selection is possible in one of three ways. (i)
‘Performance iteration’ uses the fact that at each P-IRLS step a working penalized linear model
is estimated, and the smoothing parameter estimation can be performed for each such working model.
Eventually, in most cases, both model parameter estimates and smoothing
parameter estimates converge. This option is available in bam and gamm.
(ii) Alternatively the P-IRLS scheme is iterated to convergence for each trial set of smoothing parameters,
and GCV, UBRE or REML scores are only evaluated on convergence - optimization is then ‘outer’ to the P-IRLS
loop: in this case the P-IRLS iteration has to be differentiated, to
facilitate optimization, and gam.fit3 or one of its variants is used in place of
gam.fit. (iii) The extended Fellner-Schall algorithm of Wood and Fasiolo (2017) alternates estimation of model coefficients with simple updates of smoothing parameters, eventually approximately maximizing the marginal likelihood of the model (REML). gam uses the second method, outer iteration, by default.
Several alternative basis-penalty types are built in for representing model
smooths, but alternatives can easily be added (see smooth.terms
for an overview and smooth.construct for how to add smooth classes). The choice of the basis dimension
(k in the s, te, ti and t2 terms) is something that should be considered carefully
(the exact value is not critical, but it is important not to make it restrictively small, nor very large and
computationally costly). The basis should
be chosen to be larger than is believed to be necessary to approximate the smooth function concerned.
The effective degrees of freedom for the smooth will then be controlled by the smoothing penalty on
the term, and (usually) selected automatically (with an upper limit set by
k-1 or occasionally k). Of course
the k should not be made too large, or computation will be slow (or in
extreme cases there will be more
coefficients to estimate than there are data).
Note that gam assumes a very inclusive definition of what counts as a GAM:
basically any penalized GLM can be used: to this end gam allows the non smooth model
components to be penalized via argument paraPen and allows the linear predictor to depend on
general linear functionals of smooths, via the summation convention mechanism described in
linear.functional.terms. link{family.mgcv} details what is available beyond GLMs
and the exponential family.
Details of the default underlying fitting methods are given in Wood (2011, 2004) and Wood, Pya and Saefken (2016). Some alternative methods are discussed in Wood (2000, 2017).
gam() is not a clone of Trevor Hastie's original (as supplied in S-PLUS or package gam). The major
differences are (i) that by default estimation of the
degree of smoothness of model terms is part of model fitting, (ii) a
Bayesian approach to variance estimation is employed that makes for easier
confidence interval calculation (with good coverage probabilities), (iii) that the model
can depend on any (bounded) linear functional of smooth terms, (iv) the parametric part of the model can be penalized,
(v) simple random effects can be incorporated, and
(vi) the facilities for incorporating smooths of more than one variable are
different: specifically there are no lo smooths, but instead (a) s
terms can have more than one argument, implying an isotropic smooth and (b) te,
ti or t2 smooths are
provided as an effective means for modelling smooth interactions of any
number of variables via scale invariant tensor product smooths. Splines on the sphere, Duchon splines
and Gaussian Markov Random Fields are also available. (vii) Models beyond the exponential family are available.
See package gam, for GAMs via the original Hastie and Tibshirani approach.
Value
If fit=FALSE the function returns a list G of items needed to
fit a GAM, but doesn't actually fit it.
Otherwise the function returns an object of class "gam" as described in gamObject.
WARNINGS
The default basis dimensions used for smooth terms are essentially arbitrary, and
it should be checked that they are not too small. See choose.k and
gam.check.
Automatic smoothing parameter selection is not likely to work well when
fitting models to very few response data.
For data with many zeroes clustered together in the covariate space it is quite easy to set up
GAMs which suffer from identifiability problems, particularly when using Poisson or binomial
families. The problem is that with e.g. log or logit links, mean value zero corresponds to
an infinite range on the linear predictor scale.
Author(s)
Simon N. Wood [email protected]
Front end design inspired by the S function of the same name based on the work
of Hastie and Tibshirani (1990). Underlying methods owe much to the work of
Wahba (e.g. 1990) and Gu (e.g. 2002).
References
Key References on this implementation:
Wood, S.N., N. Pya and B. Saefken (2016), Smoothing parameter and
model selection for general smooth models (with discussion).
Journal of the American Statistical Association 111, 1548-1575
doi:10.1080/01621459.2016.1180986
Wood, S.N. (2011) Fast stable restricted maximum likelihood
and marginal likelihood estimation of semiparametric generalized linear
models. Journal of the Royal Statistical Society (B) 73(1):3-36
doi:10.1111/j.1467-9868.2010.00749.x
Wood, S.N. (2004) Stable and efficient multiple smoothing parameter estimation for
generalized additive models. J. Amer. Statist. Ass. 99:673-686. [Default
method for additive case by GCV (but no longer for generalized)]
Wood, S.N. (2003) Thin plate regression splines. J.R.Statist.Soc.B 65(1):95-114
doi:10.1111/1467-9868.00374
Wood, S.N. (2006a) Low rank scale invariant tensor product smooths for
generalized additive mixed models. Biometrics 62(4):1025-1036
Wood S.N. (2017) Generalized Additive Models: An Introduction with R (2nd edition). Chapman
and Hall/CRC Press. doi:10.1201/9781315370279
Wood, S.N. and M. Fasiolo (2017) A generalized Fellner-Schall method for smoothing
parameter optimization with application to Tweedie location, scale and shape models.
Biometrics 73 (4), 1071-1081 doi:10.1111/biom.12666
Wood S.N., F. Scheipl and J.J. Faraway (2013) Straightforward intermediate rank tensor product smoothing
in mixed models. Statistics and Computing 23: 341-360. doi:10.1007/s11222-012-9314-z
Marra, G and S.N. Wood (2012) Coverage Properties of Confidence Intervals for Generalized Additive
Model Components. Scandinavian Journal of Statistics, 39(1), 53-74. doi:10.1111/j.1467-9469.2011.00760.x
Key Reference on GAMs and related models:
Wood, S. N. (2020) Inference and computation with generalized
additive models and their extensions. Test 29(2): 307-339.
doi:10.1007/s11749-020-00711-5
Hastie (1993) in Chambers and Hastie (1993) Statistical Models in S. Chapman
and Hall.
Hastie and Tibshirani (1990) Generalized Additive Models. Chapman and Hall.
Wahba (1990) Spline Models of Observational Data. SIAM
Wood, S.N. (2000) Modelling and Smoothing Parameter Estimation
with Multiple Quadratic Penalties. J.R.Statist.Soc.B 62(2):413-428 [The original
mgcv paper, but no longer the default methods.]
Background References:
Green and Silverman (1994) Nonparametric Regression and Generalized Linear Models. Chapman and Hall.
Gu and Wahba (1991) Minimizing GCV/GML scores with multiple smoothing parameters via
the Newton method. SIAM J. Sci. Statist. Comput. 12:383-398
Gu (2002) Smoothing Spline ANOVA Models, Springer.
McCullagh and Nelder (1989) Generalized Linear Models 2nd ed. Chapman & Hall.
O'Sullivan, Yandall and Raynor (1986) Automatic smoothing of regression
functions in generalized linear models.
J. Am. Statist.Ass. 81:96-103
Wood (2001) mgcv:GAMs and Generalized Ridge Regression for R. R News 1(2):20-25
Wood and Augustin (2002) GAMs with integrated model selection using penalized regression splines and applications
to environmental modelling. Ecological Modelling 157:157-177
https://www.maths.ed.ac.uk/~swood34/
See Also
mgcv-package, gamObject, gam.models, smooth.terms,
linear.functional.terms, s,
te predict.gam,
plot.gam, summary.gam, gam.side,
gam.selection, gam.control
gam.check, linear.functional.terms negbin, magic,vis.gam
Examples
library(mgcv)
set.seed(2)
dat <- gamSim(1,n=400,dist="normal",scale=2)
b <- gam(y~s(x0)+s(x1)+s(x2)+s(x3),data=dat)
summary(b)
plot(b,pages=1,residuals=TRUE)
plot(b,pages=1,seWithMean=TRUE)
gam.check(b)
G <- gam(y~s(x0)+s(x1)+s(x2)+s(x3),fit=FALSE,data=dat)
b <- gam(G=G)
print(b)
G <- gam(y~s(x0)+s(x1)+s(x2)+s(x3),fit=FALSE,data=dat,sp=b$sp)
G$lsp0 <- log(b$sp*10)
gam(G=G)
b0 <- gam(y~s(x0)+s(x1)+s(x2)+s(x3),data=dat,method="REML")
plot(b0,pages=1,scheme=1,unconditional=TRUE)
bt <- gam(y~te(x0,x1,k=7)+s(x2)+s(x3),data=dat,
method="REML")
plot(bt,pages=1)
plot(bt,pages=1,scheme=2)
AIC(b0,bt)
bt <- gam(y~s(x0)+s(x1)+s(x2)+s(x3)+ti(x1,x2,k=6),
data=dat,method="REML")
summary(bt)
bs <- gam(y~s(x0,x1,k=40)+s(x2)+s(x3),data=dat,
method="REML")
plot(bs,pages=1)
AIC(b0,bt,bs)
b1 <- gam(y~s(x0)+s(x1)+s(x2)+s(x3),data=dat,
method="REML",select=TRUE)
plot(b1,pages=1)
bp <- gam(y~s(x0)+s(x1)+s(x2)+s(x3),sp=c(0.01,-1,-1,-1),data=dat)
plot(bp,pages=1,scheme=1)
bp <- gam(y~s(x0,sp=.01)+s(x1)+s(x2)+s(x3),data=dat)
bp<-gam(y~s(x0)+s(x1)+s(x2)+s(x3),
min.sp=c(0.001,0.01,0,10),data=dat)
print(b);print(bp)
bp<-gam(y~s(x0)+s(x1)+s(x2)+s(x3),
min.sp=c(0.1,0.1,0,10),data=dat,method="REML")
print(b0);print(bp)
b0 <- gam(y~s(x0,k=4,fx=TRUE,bs="tp")+s(x1,k=12)+s(x2,k=15),data=dat)
plot(b0,pages=1)
set.seed(6)
dat <- gamSim(1,n=2000,dist="poisson",scale=.1)
b2<-gam(y~s(x0,bs="cr")+s(x1,bs="cr")+s(x2,bs="cr")+
s(x3,bs="cr"),family=poisson,data=dat,method="REML")
plot(b2,pages=1)
b2a<-gam(y~s(x0,bs="cr")+s(x1,bs="cr")+s(x2,bs="cr"),
family=poisson,data=dat,method="REML",
in.out=list(sp=b2$sp[1:3],scale=1))
par(mfrow=c(2,2))
plot(b2a)
par(mfrow=c(1,1))
dat <- gamSim(1,n=400,dist="poisson",scale=.25)
b4<-gam(y~s(x0)+s(x1)+s(x2)+s(x3),family=poisson,
data=dat,method="GACV.Cp",scale=-1)
plot(b4,pages=1)
b5<-gam(y~s(x0)+s(x1)+s(x2)+s(x3),family=poisson,
data=dat,method="REML")
plot(b5,pages=1)
dat <- gamSim(1,n=400,dist="binary",scale=.33)
lr.fit <- gam(y~s(x0)+s(x1)+s(x2)+s(x3),family=binomial,
data=dat,method="REML")
op <- par(mfrow=c(2,2))
fn <- c("f0","f1","f2","f3");xn <- c("x0","x1","x2","x3")
for (k in 1:4) {
plot(lr.fit,residuals=TRUE,select=k)
ff <- dat[[fn[k]]];xx <- dat[[xn[k]]]
ind <- sort.int(xx,index.return=TRUE)$ix
lines(xx[ind],(ff-mean(ff))[ind]*.33,col=2)
}
par(op)
anova(lr.fit)
lr.fit1 <- gam(y~s(x0)+s(x1)+s(x2),family=binomial,
data=dat,method="REML")
lr.fit2 <- gam(y~s(x1)+s(x2),family=binomial,
data=dat,method="REML")
AIC(lr.fit,lr.fit1,lr.fit2)
eg <- gamSim(2,n=500,scale=.1)
attach(eg)
op <- par(mfrow=c(2,2),mar=c(4,4,1,1))
contour(truth$x,truth$z,truth$f)
b4 <- gam(y~s(x,z),data=data)
fit1 <- matrix(predict.gam(b4,pr,se=FALSE),40,40)
contour(truth$x,truth$z,fit1)
persp(truth$x,truth$z,truth$f)
vis.gam(b4)
detach(eg)
par(op)
par(mfrow=c(2,2))
n <- 5000
eg <- gamSim(2,n=n,scale=.5)
attach(eg)
ind<-sample(1:n,200,replace=FALSE)
b5<-gam(y~s(x,z,k=40),data=data,
knots=list(x=data$x[ind],z=data$z[ind]))
vis.gam(b5,theta=30,phi=30)
plot(b5)
plot(b5,scheme=1,theta=50,phi=20)
plot(b5,scheme=2)
par(mfrow=c(1,1))
b6<-gam(y~s(x,z,k=64),data=data,knots=list(x= rep((1:8-0.5)/8,8),
z=rep((1:8-0.5)/8,rep(8,8))))
vis.gam(b6,color="heat",theta=30,phi=30)
b7 <- gam(y~s(x,z,k=40,xt=list(max.knots=500,seed=2)),data=data)
vis.gam(b7,theta=30,phi=30)
detach(eg)