An easy way to do this is to use a Format expression instead of accessing your report field directly.
e.g.
=Format(Fields!OrderPrice.Value, "#,###.00")
instead of..
=Fields!OrderPrice.Value
This will coerce the numeric value into a string. The number format will ensure you keep the comma. When exported via CSV, the value will be wrapped in quotes, meaning your formatting comma won't trip a parser up.
Massive caveat
This solution is not perfect. The problem is that although you get the desired result in your CSV file, any SSRS Excel export you produce will flag that field as textual content, meaning that your users will have to tell Excel different every time they open that report.
Similarly, if someone opens your CSV file in Excel, the same thing will happen. Excel isn't smart enough to recognise that the thing in quotes is actually a number, and it'll again, treat the column as textual.
The reality is that CSV is a limited format. It's primary concern is content - it doesn't convey formatting information, and even if it were possible to give SSRS more "guidance" on individual export routines, the inherent limitations of the CSV file format mean that some sort of compromise ( or extra work on the Excel end ) has to be made.