Better algorithm to construct a matrix using vcat() or hcat() in a loop in Julia?

Question

To use vcat(a,b) and hcat(a,b), one must match the number of columns or number of rows in the matrices a and b.

When constructing a matrix using vact(a, b) or hcat(a, b) in a loop, one needs an initial matrix a (like a starting statement). Although all the sub-matrices are created in the same manner, I might need to construct this initial matrix a outside of the loop.

For example, if the loop condition is for i in 1:w, then I would need to pre-create a using i = 1, then start the loop with for i in 2:w.

If there is a nested loop, then my method is very awkward. I have thought the following methods, but it seems they don't really work:

Use a dummy a, delete a after the loop. From this question, we cannot delete row in a matrix. If we use another variable to refer to the useful rows and columns, we might waste some memory allocation.

Use reshape() to make an empty dummy a. It works for 1 dimension, but not multiple dimensions.

julia> a = reshape([], 2, 0)
2×0 Array{Any,2}

julia> b = hcat(a, [3, 3])
2×1 Array{Any,2}:
 3
 3

julia> a = reshape([], 2, 2)
ERROR: DimensionMismatch("new dimensions (2,2) must be consistent with array size 0")
 in reshape(::Array{Any,1}, ::Tuple{Int64,Int64}) at ./array.jl:113
 in reshape(::Array{Any,1}, ::Int64, ::Int64, ::Vararg{Int64,N}) at ./reshapedarray.jl:39

So my question is how to work around with vcat() and hcat() in a loop?

Edit: Here is the problem I got stuck in:

There are many gray pixel images. Each one is represented as a 20 by 20 Float64 array. One function foo(n) randomly picks n of those matrices, and combine them to a big square.

If n has integer square root, then foo(n) returns a sqrt(n) * 20 by sqrt(n) * 20 matrix.

If n does not have integer square root, then foo(n) returns a ceil(sqrt(n)) * 20 by ceil(sqrt(n)) * 20 matrix. On the last row of the big square image (a row of 20 by 20 matrices), foo(n) fills ceil(sqrt(n)) ^ 2 - n extra black images (each one is represented as zeros(20,20)).

My current algorithm for foo(n) is to use a nested loop. In the inner loop, hcat() builds a layer (consisting ceil(sqrt(n)) images). In the outer loop, vcat() combines those layers.

Then dealing with hcat() and vcat() in a loop becomes complicated.

Assuming you can't preallocate, a common trick in this situation is to avoid vcat or hcat entirely by building a Vector{Vector{T}} dynamically by using push! or append!, and then convert the result to a matrix when you're done (the final conversion is unlikely to have a performance impact on your routine except in unusual cases). I suspect this will be significantly faster than looping over calls to vcat as long as you're careful to take advantage of column-major order... — Colin T Bowers
@ColinTBowers If I am building a big Multi-dimensional matrix from many small Multi-dimensional matrices, it might be complicated to compress each small matrix then apply push! or append! to each element of that matrix. Are there better ways to deal with this situation? — Jay Wong
The common julia syntax is to preallocate the array, then fill it in the loop. Is that not possible here? — Michael K. Borregaard
Could you give a minimal example of the kind of thing you want to do? — David P. Sanders
@MichaelK.Borregaard I just added an example, I am not sure how to preallocate the big square matrix and then fill it. — Jay Wong

Michael K. Borregaard Michael K. Borregaard · Accepted Answer · 2017-01-11T21:32:21

So would:

pickimage() = randn(20,20)
n = 16
m = ceil(Int, sqrt(n))
out = Matrix{Float64}(20m, 20m)
k = 0
for i in (1:m)-1
   for j in (1:m)-1
       out[20i + (1:20), 20j + (1:20)] .= ((k += 1) <= n) ? pickimage() : zeros(20,20)
   end
end

be a relevant solution?

Better algorithm to construct a matrix using vcat() or hcat() in a loop in Julia?

1 Answers