The original sin is
function f = test(n)
(...)
f(n) = (...)
(...)
endfunction
f is supposed to be the result of the function. Therefore, f(n) is not "the value that the function test takes on argument n", but "the n-th element of f". Scilab then handles this however it can; on your test case, it tries to access a non-integer index, which results in an error. Your loop solution solves the problem.
Replacing all three f(n) by f in your first formulation makes it into something that works... as long as the argument is a scalar (not an array).
If you want test to be able to accept vector arguments without making a loop, the problem is that n < 0 is a vector of the same size as n. My solution would use logical arrays for indexing each of the three conditions:
function f = test(n)
f = zeros(size(n));
negative = (n<0);//parentheses are optional, but I like them for readability
greater_than_1 = (n>1);
others = ~negative & ~greater_than_1;
f(isnegative)=0;
f(greater_than_1)=1;
f(others) = sin(%pi/2*n(others));
endfunction