Hey, I just started with the book and I have a question on formula 2.68. Here the constraint
is stated. But this would mean that D has to be a orthogonal matrix with n=l. And this would miss the whole point of the compression. At the beginning of section 2.12 it is explained that l<n, columns of D are orthogonal and
Since shape(D) is (n,l), and transpose of D is (l,n), after left multiply D with its transpose, the output shape will be(l,l), and the constraint of all columns of D are orthonormal is essential if you wanna simplify the matrix myltiplication
Read more… (41 words)
Read more (41 words)
Chapter 2: Linear Algebra
If you're reading Deep Learning Book, feel free to ask questions, discuss the material or share resources.
Below is a video from a book club in San Francisco, USA discussing this chapter. Presented by Gavin Crooks.
I've always wondered about deep learning's history, and had read in several articles that neural networks and back-propagation used to be popular in 1980s. Didn't know that there was another deep learning phase in the 1960s!