This is a brief summary of ML course provided by Andrew Ng and Stanford in Coursera.
You can find the lecture video and additional materials in
https://www.coursera.org/learn/machine-learning/home/welcome
Coursera | Online Courses From Top Universities. Join for Free
1000+ courses from schools like Stanford and Yale - no application required. Build career skills in data science, computer science, business, and more.
www.coursera.org
Matrix multiplication is very useful, since you can pack a lot of computation into just one matrix multiplication operation.
Commutative
When working with scalars(real numbers), it has commutative properties.
e.g., 3 X 5 = 5 X 3
But this is not true for matrices multiplications.
Let A and B be matrices. Then in general, A X B $\neq$ B X A (not commutative)
e.g., $\left[\begin{array} {rrr} 1 & 1 \\ 0 & 0 \end{array}\right] $ $\left[\begin{array} {rrr} 0 & 0 \\ 2 & 0 \end{array}\right] $ = $\left[\begin{array} {rrr} 2 & 0 \\ 0 & 0 \end{array}\right] $
e.g., $\left[\begin{array} {rrr} 0 & 0 \\ 2 & 0 \end{array}\right] $ $\left[\begin{array} {rrr} 1 & 1 \\ 0 & 0 \end{array}\right] $ = $\left[\begin{array} {rrr} 0 & 0 \\ 2 & 2\end{array}\right] $
Associative
When working with scalars(real numbers), it has associative properties.
e.g., (3 X 5) X 2 = 3 X (5 X 2)
This is same for matrices
e.g., (A X B) X C = A X (B X C)
Identity Matrix
When working with scalars(real numbers), 1 is identity.
e.g., 1 X z = z X 1 = z, for any z
For matrix, 1 goes along the diagonal and 0 for everywhere else.
For any matrix A,
A * I = I * A = A
Denoted I (or $I_{nxn}).
Examples of identity matrices:
e.g., $\left[\begin{array} {rrr} 1 & 0 \\ 0 & 1 \end{array}\right] $ $\left[\begin{array} {rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] $ $\left[\begin{array} {rrr} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1\end{array}\right] $
Quiz: what is $\left[\begin{array} {rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] $ $\left[\begin{array} {rrr} 1 \\ 3 \\ 2 \end{array}\right] $ ?
Answer: $\left[\begin{array} {rrr} 1 \\ 3 \\ 2 \end{array}\right] $
Lecturer's Note
- Matrices are not commutative: A∗B≠B∗A
- Matrices are associative: (A∗B)∗C=A∗(B∗C)
The identity matrix, when multiplied by any matrix of the same dimensions, results in the original matrix. It's just like multiplying numbers by 1. The identity matrix simply has 1's on the diagonal (upper left to lower right diagonal) and 0's elsewhere.
$\left[\begin{array} {rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right] $
When multiplying the identity matrix after some matrix (A∗I), the square identity matrix's dimension should match the other matrix's columns. When multiplying the identity matrix before some other matrix (I∗A), the square identity matrix's dimension should match the other matrix's rows.
