1.3 Power Transforms

In this section, you learn how to:
  • Symmetrize a skewed distribution using a logarithmic transformation
  • Select a member of the Box-Cox family of transform

Video Overview of the Section (Alternative .mp4 Version – 10:20 min)

In the Section 1.2 example, we considered claims without justifying the use of the logarithmic scaling. When analyzing variables such as assets of firms, wages of individuals and housing prices of households in business and economic applications, it is common to consider logarithmic instead of the original units. A log transform retains the original ordering (for example, large wages remain large on the log wage scale) but serves to “pull in” extreme values of the distribution.

To illustrate, Figure 1.6 shows the bodily injury claims distribution in (thousands of) dollars. In order to graph the data meaningfully, the largest observation ($50,000) was removed prior to making this plot. Even with this observation removed, Figure 1.6 shows that the distribution is heavily lop-sided to the right, with several large values of claims appearing.

Distributions that are lopsided in one direction or the other are known as skewed. Figure 1.6 is an example of a distribution skewed to the right, or positively skewed. Here, the tail of the distribution on the right is longer and there is a greater concentration of mass to the left. In contrast, a left-skewed, or negatively skewed distribution, has a longer tail on the left and a greater concentration of mass to the right. Many insurance claims distributions are right-skewed (see the text by Klugman, Panjer and Willmot, 2008, for extensive discussions). As we saw in Figures 1.4 and 1.5, a logarithmic transformation yields a distribution that is only mildly skewed to the left.

F1BIOrig

Figure 1.6. Distribution of Bodily Injury Claims. Observations are in (thousands of) dollars with the largest observation omitted.

R Code for Figure 1.6

Logarithmic transformations are used extensively in applied statistics work. One advantage is that they serve to symmetrize distributions that are skewed. More generally, we consider power transforms, also known as the Box-Cox family of transforms. Within this family of transforms, in lieu of using the response $y$, we use a transformed, or rescaled version, $y^{\lambda}$. Here, the power $\lambda$ ($\lambda$, a Greek “el”) is a number that may be user specified. Typical values of $\lambda$ that are used in practice are $\lambda =1, 1/2, 0$ or $-1$. When we use $\lambda =0$, we mean $\ln (y)$, that is, the natural logarithmic transform. More formally, the Box-Cox family can be expressed as
$$
y^{(\lambda)}=\left\{ \begin{array}{ll} \frac{y^{\lambda }-1}{\lambda} & \lambda \neq 0 \\ \ln (y) & \lambda =0 \end{array} \right. .
$$
As we will see, because regression estimates are not affected by location and scale shifts, in practice we do not need to subtract one nor divide by $\lambda$ when rescaling the response. The advantage of the above expression is that, if we let $\lambda$ approach 0, then $y^{(\lambda)}$ approaches $\ln (y)$, from some straightforward calculus arguments.

To illustrate the usefulness of transformations, we simulated 500 observations from a chi-square distribution with two degrees of freedom. Appendix A3.2 introduces this distribution (that we will encounter again later in studying the behavior of test statistics). The upper left panel of Figure 1.7 shows the original distribution is heavily skewed to the right. The other panels in Figure 1.7 show the data rescaled using the square root, logarithmic and negative reciprocal transformations. The logarithmic transformation, in the lower left panel, provides the best approximation to symmetry for this example. The negative reciprocal transformation is based on $\lambda =-1$, and then multiplying the rescaled observations by minus one, so that large observations remain large.

F1Chisquare

Figure 1.7. 500 simulated observations from a chi-square distribution. The upper left panel is based on the original distribution. The upper right corresponds to the square root transform, the lower left to the log transform and the lower right to the negative reciprocal transform.

R Code for Figure 1.7

[raw] [/raw]