I've always wondered about deep learning's history, and had read in several articles that neural networks and back-propagation used to be popular in 1980s. Didn't know that there was another deep learning phase in the 1960s!
Consider what happens when all the xi are equal to some constantc. Analytically,we can see that all the outputs should be...
Read more… (45 words)
Read (45 words)
Question on formula 2.68
Hey, I just started with the book and I have a question on formula 2.68. Here the constraint
is stated. But this would mean that D has to be a orthogonal matrix with n=l. And this would miss the whole point of the compression. At the beginning of section 2.12 it is explained that l<n, columns of D are orthogonal and
Since shape(D) is (n,l), and transpose of D is (l,n), after left multiply D with its transpose, the output shape will be(l,l), and the constraint of all columns of D are orthonormal is essential if you wanna simplify the matrix myltiplication