Verilog generate block are used to describe physical hardware. As such, an inifinite loop in a generate block will require infinite resources.
Any for loop inside a generate statement must be of a fixed and finite size that can be determined during synthesis.
Remember that HDL is not executed sequentially, but describes connections between physical circuits. Since it appears that you only require one instance of the calb_top module, you don't require either the generate block or the for loop.
Edit:
Since you're intending to perform an iterative process, you have two options, as Greg pointed out in his comment below - you can either instantiate a fixed number of calb_top blocks (since an infinite number would require an infinite amount of space) or to re-use the same block some number of times.
Here are some samples. I've haven't sim'd or synthesized them, but they're logically correct.
N-Block solution
module top(a1,a3,wj,d4,d10,d2,dc,dtot,clock,done);
parameter NUM_BLOCKS = 10;
input [11:0]a1,a3,wj;
input [3:0]d4;
input [9:0]d10;
input [1:0]d2;
input [25:0]dc;
output [25:0]dtot;
wire [11:0] a1s [NUM_BLOCKS:0];
wire [11:0] a3s [NUM_BLOCKS:0];
wire [25:0] dt [NUM_BLOCKS-1:0];
wire [25:0] error [NUM_BLOCKS-1:0];
assign a1s[0]=a1;
assign a3s[0]=a3;
genvar i;
generate
for (i=0;i<NUM_BLOCKS;i=i+1)begin:test
calb_top t1(a1s[i],a3s[i],wj,d4,d10,d2,dc,dt[i],error[i],a1s[i+1],a3s[i+1]);
end
endgenerate
assign dtot=dt[NUM_BLOCKS-1];
endmodule
This links together a number of calb_top blocks equal to NUM_BLOCKS, then outputs the result of the final block to dtot. This doesn't do any checks on the error, so you may want to put in your own code to check error[NUM_BLOCKS-1] (the error of the final calb_top).
Single-Block solution:
module top(clock,start,a1,a3,wj,d4,d10,d2,dc,dtot);
input clock;
input start;
input [11:0]a1,a3,wj;
input [3:0]d4;
input [9:0]d10;
input [1:0]d2;
input [25:0]dc;
output reg[25:0]dtot;
wire [25:0]dt,error;
reg [11:0] a1in, a3in;
wire [11:0] alpha1,alpha3;
calb_top t1(a1in,a3in,wj,d4,d10,d2,dc,dt,error,alpha1,alpha3);
always @(posedge clock)
begin
if (start)
begin
a1in <= a1;
a3in <= a3;
end
else
begin
a1in <= alpha1;
a3in <= alpha3;
end
end
always @(posedge clock)
if (start)
dtot <= 0;
else if (error == 0)
dtot <= dt;
else
dtot <= dtot;
endmodule
Each clock cycle, we run one pass through calb_top. If start is 1, then a1 and a3 are used as inputs. Otherwise, the previous outputs alpha1 and alpha3 are used. When error is 0, then dtot is set. Note that I've added clock and start to the port list.