When you find yourself building an "extremely complex SQL statement", it's usually best to step back and rethink.
Remember this - the vast majority of operations performed on database table are selects, not inserts or updates (though there are exceptions to every rule, of course).
The right time to be "spending" CPU cycles calculating things like author lists is when the list changes, not when you just want to extract the information.
Add another column to the book table called author_list and then create an insert/update trigger on authors so that this column is rebuilt whenever an author is changed for the specific ISBN.
That puts the cost where it should be and will make your query a lot simpler. The trigger ensures the data stays consistent, and it's okay to break 3NF if you know what you're doing.
As to the subcategory, the case
statement can be your friend, but per-row functions on select never scale well.
I would just create a set of rows in subcategories with the id of 0 (one for each category) and make its name blank. Then it can be done with a simple join without having to concern yourself with performance. This could also be don with a trigger on category so every category will always have a subcategory of 0.
With those two changes, the query becomes a lot less complex, something along the lines of:
select b.isbn, b.title, b.author_list, c.name, sc.name, b.price
from Book b, Category c, SubCategory sc
where b.category_id = c.category_id
and b.category_id = sc.category_id
and b.subcategory_id = sc.subcategory_id
order by ...
This query should scream along since it's using just the basic levels of relational algebra (i.e., no per-row functions (including case statements), no subqueries). And that's an "old-school" query, you may get even more performance by using explicit rather than implicit JOINs.
One final point: a properly 3NF schema would not have the ISBN in the authors table - a better option would be to have a separate BookAuthor table holding the ISBN and author_id to properly model the many-to-many relationship. But you may have aleready changed that for performance (I don't know).