13.1 Introduction

There are many ways to extend, or generalize, the linear regression model. This chapter introduces an extension that is so widely used that it is known as the “generalized linear model,” or as the acronym GLM.

Generalized linear models include linear, logistic and Poisson regressions, all as special cases. One common feature of these models is that in each case we can express the mean response as a function of linear combinations of explanatory variables. In the GLM context, it is customary to use (mu_i = mathrm{E}~y_i) for the mean response and call (eta_i = mathbf{x}_i^{mathbf{prime}} boldsymbol beta) the systematic component of the model. We have seen that we can express the systematic component as:

• (mathbf{x}_i^{mathbf{prime}} boldsymbol beta = mu_i), for (normal) linear regression,
• (mathbf{x}_i^{mathbf{prime}} boldsymbol beta = exp(mu_i)/(1+exp(mu_i)),) for logistic regression and
• (mathbf{x}_i^{mathbf{prime}} boldsymbol beta = ln (mu_i),) for Poisson regression.

For GLMs, the systematic component is related to the mean through the expression
begin{equation}
eta _i = mathbf{x}_i^{mathbf{prime}} boldsymbol beta = mathrm{g}left( mu _iright).
end{equation} Here, g(.) is known and called the link function. The inverse of the link function, (mu _i = mathrm{g}^{-1}( mathbf{x}_i^{mathbf{prime}} boldsymbol beta)), is the mean function.

The second common feature involves the distribution of the dependent variables. In Section 13.2, we will introduce the linear exponential family of distributions, an extension of the exponential distribution. This family includes the normal, Bernoulli and Poisson distributions as special cases.

The third common feature of GLM models is the robustness of inference to the choice of distributions. Although linear regression is motivated by normal distribution theory, we have seen that responses need not be normally distributed for statistical inference procedures to be effective. The Section 3.2 sampling assumptions focus on:

• the form of the mean function (assumption F1),
• non-stochastic or exogenous explanatory variables (F2),
• constant variance (F3) and
• independence among observations (F4).

GLM models maintain assumptions F2 and F4 and generalize F1 through the link function. The choice of different distributions allows us to relax F3 by specifying the variance to be a function of the mean, written as (mathrm{Var~}y_i = phi v(mu_i)). Table 13.1 shows how the variance depends on the mean for different distributions. As we will see when considering estimation (Section 13.3), it is the choice of the variance function that drives the most important inference properties, not the choice of the distribution.

begin{matrix}
begin{array}{c}
text{Table 13.1. Variance Functions for Selected Distributions}
end{array}\small
begin{array}{lc} hline
text{Distribution} & text{Variance Function} v(mu) \ hline
text{Normal} & 1 \
text{Bernoulli} & mu ( 1- mu ) \
text{Poisson} & mu \
text{Gamma} & mu ^2 \
text{Inverse Gaussian} & mu ^3 \ hline
end{array}
end{matrix}

By considering regression in the GLM context, we will be able to handle dependent variables that are approximately normally distributed, binary or representing counts, all within one framework. This will aid our understanding of regression by allowing us to see the “big picture” and not be so concerned with the details. Further, the generality of GLMs will allow us to introduce new applications, such as gamma regressions that are useful for fat-tailed distributions and the so-called “Tweedie” distributions for two-part data. Two-part data is a topic taken up in Chapter 16, where there is a mass at zero and a continuous component. For insurance claims data, the zero represents no claim and the continuous component represents the amount of a claim.

This chapter describes estimation procedures for calibrating GLM models, significance tests and goodness of fit statistics for documenting the usefulness of the model, and residuals for assessing the robustness of the model fit. We will see that the our earlier work done on linear, binary and count regression models provides the foundations for the tools needed for the GLM model. Indeed, many are slight variations of tools and concepts developed earlier in this text and we will be able to build on these foundations.