Here we consider the problem of approximately factorising a matrix without constraints and show that solutions can be generated from the orthonormal eigenvectors of the Gram matrix (i.e. of the sample covariance matrix).
For this we need the eigendecomposition of real symmetric matrices.
Questions, all related to one another:
- What other solutions are there?
- (Speculative) can we characterise the solutions as orbits of the orthogonal group on the solutions above, and on those solutions obtained from the above by adding rows of zeros to ?
- Under what constraints, if any, are the optimal solutions to matrix factorisation matrices with orthonormal rows/columns? To what extent does orthogonality come for free?