You should read The Reasoned Schemer for ideas. Basically the way to do math in a logic program is to create list-based encodings of numbers, which the logic engine can grow as needed to try things out. I don't have the book handy, but it encodes integers as a list of bits, in some weird way I can't quite recall: maybe (1) represents 0, (0) is illegal, and the MSB is last in the list?
Anyway, that's a lot of work; David Nolen has also recently introduced something about finite domains into core.logic. I don't know how those work, but I think they simplify the problem a lot for you by letting you specify what kinds of numbers to consider as a solution to your problem.