PCA results are affected by the units of the variables. Apart from that, if some variable's variance is much greater than the others, that variable tends to coincide with the first principal component.
A way to overcome those problems is using correlation instead of covariance matrix - provided that the differences in variances do not contain valuable information for the problem in hand.
The previous stand for FA also, if the type of factoring is "principal components". Conversely, if you use "maximum likelihood" factoring, the choice of either covariance or correlation matrix does not affect the results.