TensorFlow Convolution code Optimization

Question

I am using C++ version of TensorFLow and have built 'TensorFlow for Android' successfully using below command 'bazel build -c opt //tensorflow/examples/android:tensorflow_demo' as described in https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android#bazel

I am trying to optimize the convolution code. Below are the issues faced

Unable to find the exact location of convolution code. I am able to debug the code till below function in

'return choose( Cond(), kernel.reshape(kernel_dims) .contract(input .extract_image_patches( kernelRows, kernelCols, row_stride, col_stride, row_in_stride, col_in_stride, padding_type) .reshape(pre_contract_dims), contract_dims) .reshape(post_contract_dims), input .extract_image_patches(kernelRows, kernelCols, row_stride, col_stride, row_in_stride, col_in_stride, padding_type) .reshape(pre_contract_dims) .contract(kernel.reshape(kernel_dims), contract_dims) .reshape(post_contract_dims));'

as present in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/eigen_spatial_convolutions.h

I have few questions related to the above function.

1.1 Is the above function really performing convolution ? If so where is the code ?

1.2 Is contraction (contract function ) same as convolution ? If both convolution and contraction are same, why is the contract operation being performed to both input and kernel matrix ?

1.3 Where are the definitions of functions - choose, reshape, contract ,extract image patches etc ?

2.Unable to extract data (matrices ) from input and kernel matrix .This is in reference to the same page in the above link

2.1 I have found a line of code 'kern(kernel);' at line no 946 in the above page. Can I know the location definition of the above function ?

2.2 I am unable to extract input and kernel matrices from the corresponding 4d tensors(input and kernel) as a float array, as i would like to try optimizing the convolution code using parallel processing. I couldn't find any method to convert Tensor Matrices from Tensor 4D to an array.

Please help me in answering the above questions

2.1) kern is an object with type TensorRef and kern(Kernel) calls the constructor of that object with input kernel. — Hamed

Nek Nek · Accepted Answer · 2018-12-03T20:29:46

1.1) Yes it is, the code is what comes after the Cond() statement:

// If this condition is met, the first argument is chosen, if not, the second one
// is chosen, this condition checks if the input is ColMajor or RowMajor, all of
// the tests I've done result in a RowMajor but I don't know what determines this
// exactly
return choose( Cond(),

// ColMajor
kernel.reshape(kernel_dims) .contract(input 
.extract_image_patches( kernelRows, kernelCols, row_stride, col_stride, 
row_in_stride, col_in_stride, padding_type) .reshape(pre_contract_dims), 
contract_dims) .reshape(post_contract_dims),

// RowMajor
input.extract_image_patches(kernelRows, kernelCols, row_stride, col_stride, 
row_in_stride, col_in_stride, padding_type) .reshape(pre_contract_dims) 
.contract(kernel.reshape(kernel_dims),contract_dims).reshape(post_contract_dims));

1.2) No, contraction is an abstraction of matrix multiplication to Tensors of N dimensions. It is only being applyed to one of them, depending on the condition

1.3) These are all Eigen functions, Eigen has a rather not helpfull documentation of their Tensor operations.I found this on their wiki which can help you to understand what they do, it is not thorough but it can help you wrap your head around the idea of the operation.

2.1) I don't know where it is either.

2.2) I'm not sure if this can be done directly, Eigen's functions can be rather unintuitive, thou if you know the shape of the 4D tensor, you can create a matrix and just assign every element to that matrix (which I reckon wouldn't be very efficient)

I just realized this was posted a year ago but I had already wrote my answer, it might be useful to someone else so I'll just leave it here.

TensorFlow Convolution code Optimization

2 Answers