Matrix Multiplication

In general, if (mathbf{A}) is a matrix of dimension (ntimes c) and ( mathbf{B}) is a matrix of dimension (ctimes k), then (mathbf{C}=mathbf{AB }) is a matrix of dimension (ntimes k) and is defined by
begin{equation*} mathbf{C}=mathbf{AB=}left( sum_{s=1}^{c}a_{is}b_{sj}right) _{ij}. end{equation*} For example consider the (2times 2) matrices
begin{equation*} mathbf{A}=left( begin{array}{cc} 2 & 5 \ 4 & 1 end{array} right) text{ }mathbf{B}=left( begin{array}{cc} 4 & 6 \ 8 & 1 end{array} right) . end{equation*} The matrix (mathbf{AB}) has dimension (2times 2). To illustrate the calculation, consider the number in the first row and second column of ( mathbf{AB}). By the rule presented above, with (i=1) and (j=2), the corresponding element of (mathbf{AB}) is ( sum_{s=1}^2a_{1s}b_{s2}=a_{11}b_{12}+a_{12}b_{22}=2(6)+5(1)=17). The other calculations are summarized as
begin{equation*} mathbf{AB}=left( begin{array}{cc} 2(4)+5(8) & 2(6)+5(1) \ 4(4)+1(8) & 4(6)+1(1) end{array} right) =left( begin{array}{cc} 48 & 17 \ 24 & 25 end{array} right) . end{equation*} As another example, suppose
begin{equation*} mathbf{A}=left( begin{array}{ccc} 1 & 2 & 4 \ 0 & 5 & 8 end{array} right) text{ }mathbf{B}=left( begin{array}{c} 3 \ 5 \ 2 end{array} right) . end{equation*} Because (mathbf{A}) has dimension (2times 3) and (mathbf{B}) has dimension (3times 1), this means that the product (mathbf{AB}) has dimension (2times 1).. The calculations are summarized as
begin{equation*} mathbf{AB}=left( begin{array}{c} 1(3)+2(5)+4(2) \ 0(3)+5(5)+(2) end{array} right) =left( begin{array}{c} 21 \ 41 end{array} right) . end{equation*} For some additional examples, we have begin{equation*} left( begin{array}{cc} 4 & 2 \ 5 & 8 end{array} right) left( begin{array}{c} a_1 \ a_2 end{array} right) =left( begin{array}{c} 4a_1+2a_2 \ 5a_1+8a_2 end{array} right) . end{equation*} begin{equation*} left( begin{array}{ccc} 2 & 3 & 5 end{array} right) left( begin{array}{c} 2 \ 3 \ 5 end{array} right) =2^2+3^2+5^2=38text{ }left( begin{array}{c} 2 \ 3 \ 5 end{array} right) left( begin{array}{ccc} 2 & 3 & 5 end{array} right) =left( begin{array}{ccc} 4 & 6 & 10 \ 6 & 9 & 15 \ 10 & 15 & 25 end{array} right) . end{equation*} In general, you see that (mathbf{ABneq BA}) in matrix multiplication, unlike multiplication of scalars (real numbers). Further, we remark that the identity matrix serves the role of “one” in matrix multiplication, in that (mathbf{AI=A}) and (mathbf{IA=A}) for any matrix (mathbf{A}), providing that the dimensions are compatible to allow matrix multiplication.

Basic Linear Regression Example of Matrix Multiplication. Define
begin{equation*} mathbf{X}=left( begin{array}{cc} 1 & x_1 \ vdots & vdots \ 1 & x_n end{array} right) text{ and }boldsymbol beta =left( begin{array}{c} beta_0 \ beta_1 end{array} right) text{, to get } mathbf X boldsymbol beta =left( text{ } begin{array}{c} beta_0+beta_1x_1 \ vdots \ beta_0+beta_1x_n end{array} right) =mathbf{mathrm{E~}mathbf{y.}} end{equation*} Thus, this yields the familiar matrix expression of the regression model, ( mathbf{y=X } boldsymbol beta + boldsymbol varepsilon .) Other useful quantities include
begin{equation*} mathbf{y}^{prime }mathbf{y=}left( begin{array}{ccc} y_1 & cdots & y_n end{array} right) left( begin{array}{c} y_1 \ vdots \ y_n end{array} right) =y_1^2+cdots +y_n^2=sum_{i=1}^{n}y_i^2, end{equation*}
begin{equation*} mathbf{X}^{prime }mathbf{y=}left( begin{array}{ccc} 1 & cdots & 1 \ x_1 & cdots & x_n end{array} right) left( begin{array}{c} y_1 \ vdots \ y_n end{array} right) =left( begin{array}{c} sum_{i=1}^{n}y_i \ sum_{i=1}^{n}x_iy_i end{array} right) end{equation*} and
begin{equation*} mathbf{X}^{prime }mathbf{X=}left( begin{array}{ccc} 1 & cdots & 1 \ x_1 & cdots & x_n end{array} right) left( begin{array}{cc} 1 & x_1 \ vdots & vdots \ 1 & x_n end{array} right) =left( begin{array}{cc} n & sum_{i=1}^{n}x_i \ sum_{i=1}^{n}x_i & sum_{i=1}^{n} x_i^2 end{array} right) . end{equation*} Note that (mathbf{X}^{prime }mathbf{X}) is a symmetric matrix.

[raw] [/raw]