Using FLoops for parallelization in julia

Question

I am trying to run this loop using floops for parallelization. However, I am unable to push or append the data to an array.

a=[]
b=[]
@floop for i in 1:10
    @reduce() do (r_df=filter(row-> row.col1==1, df))
    result=mean(r_df.col3)
    push!(a, result)
    push!(b, i)
    end
end

I tried using @reduce macro on the appending the data but it doesn't work. Any suggestion how to make this loop work? Thanks in advance!

Oscar Smith Oscar Smith · Accepted Answer · 2021-01-18T05:51:13

The problem is multi-threaded pushing into an array can't be done in parallel since parallel means it can happen in any order. What you want is

a=zeros(10)
b=zeros(10)
@floop for i in 1:10
    @reduce() do (r_df=filter(row-> row.col1==1, df))
    result=mean(r_df.col3)
    a[i] = result
    b[i] = i
    end
end

Using FLoops for parallelization in julia

1 Answers