I simply want to do: C_i=\Sum_k (A_i -B_k)^2
I saw that this calculation is faster with a simple for loop than with the numpy.subtract.outer! Anyway I feel that numpy.einsum would be the fastest. I could not understand numpy.einsum that well. Can anyone please help me out? Additionally, it would be great if someone explains how a general summation expression consisting of vector/matrices can be written with numpy.einsum?
I did not find solution for this particular problem on the web. Sorry if duplicate in some manner.
MWE with loop and numpy.subtract.outer--
A)With loop
import timeit
code1="""
import numpy as np
N=10000;
a=np.random.rand(N); b=10*(np.random.rand(N)-0.5);
def A1(x,y):
Nx=len(x)
z=np.zeros(Nx)
for i in np.arange(Nx):
z[i]=np.sum((x[i]-y)*(x[i]-y))
return z
C1=A1(a,b)"""
elapsed_time = timeit.timeit(code1, number=10)/10
print "time=", elapsed_time
B) With numpy.subtract.outer
import timeit
code1="""
import numpy as np
N=10000;
a=np.random.rand(N); b=10*(np.random.rand(N)-0.5);
def A2(x,y):
C=np.subtract.outer(x,y);
return np.sum(C*C, axis=1)
C2=A2(a,b)"""
elapsed_time = timeit.timeit(code1, number=10)/10
print "time=", elapsed_time
For N=10000 the loop becomes faster. For N=100, the outer subtract becomes faster. For N=10^5, outer subtract faces memory issue on my desktop with 8GB ram!
einsumis a generalization of the matrix product. With a general set of dimensions is performs a sum of products on selected dimensions. In other words it isC_? = sum_? A_? * B_?. - hpauljeinsumis multiplication. But still can we do the above summation with it? If not, what would be the optimised solution for these cases?forloop in python is very slow compared to that of C. - deltasataAandB? - Warren Weckessersawthe loop is faster than usingsubtract.outer? where? Can you demonstrate? I would also test a variation onouterusing broadcasting. - hpaulj